Balancing Act: Early, Fair, and Accurate Identification of At-Risk Students

Abstract

Machine learning algorithms have been widely used for identifying at-risk students. Current research focuses on timeliness and accuracy of the predictions, leading to a heavy reliance on demographic data, which introduces severe bias issues. This study develops fairness-aware machine learning models to identify at-risk students in high school Advanced Placement (AP) statistics, where student performance is closely linked to demographic background. We evaluated the predictive performance and bias mitigation strategies of various machine learning algorithms. To determine the optimal time for accurate and fair identification of at-risk students, we divided the dataset into three stages corresponding to the course’s progress. At each stage, we examined model performance and fairness across groups defined by race, gender, and eligibility for free/reduced-price lunch. Our findings suggest that at Stage 1 (i.e., up to the first unit review assignment), the models effectively identified at-risk students while maintaining fairness across demographic groups. We discovered that incorporating more learning activity data reduced the potential bias caused by overreliance on demographic information. We also examined the impact of different bias mitigation approaches as well as the exclusion of the sensitive features on predictive accuracy and fairness. We further discuss their implications for designing more context-specific solutions in educational settings.