Detecção de drift nos microdados dos cursos de computação no Enade

Abstract

Large-scale assessments are nationwide exams designed to measure the performance of educational institutions in relation to their students’ level. The Enade is one such assessment, conducted to evaluate higher education programs. This exam provides open-format microdata related to students’ socioeconomic aspects, allowing researchers to investigate the relationship between a course’s socioeconomic context and its performance in Enade. Such research can lead to advancements in higher education and enhance the value of large-scale assessments. Enade has been conducted since 2004, and after years of implementation, a large volume of data has been accumulated, which has been explored in studies in the field of Machine Learning. In this field, there is a phenomenon known as model drift, where the performance of learning models is affected due to the dynamic nature of real-world data, including socioeconomic and political factors that frequently change over time. This change in data nature is called drift and is classified into two categories: concept drift and covariate shift. Concept drift refers to a change in the relationship between the input variables and the model’s output variable, while covariate shift refers to significant changes in the distribution of input variables or the relationship between them. Considering various studies that apply learning algorithms to Enade’s microdata, the poor performance achieved by the models, and the large amount of exam data that may present drift, this study evaluates whether the low performance of models can be explained using drift detection techniques. For this, a dataset of computing-related courses was compiled, and two covariate shift detection algorithms were applied: the statistical Kolmogorov-Smirnov Test and a learning-based algorithm using the Light Gradient-Boosting Machine model. Additionally, the Reverse Concept Drift algorithm was used to detect concept drift. Based on the results of these algorithms, a relationship was established between historical events that impacted education and the socioeconomic variables in which drift was identified. As a result, multiple cases of covariate shift and indications of concept drift were detected in Enade’s microdata, explaining the poor performance of models on the dataset.