Learning Analytics for Early Identification of At-Risk Students and Feedback Intervention

Abstract

Supporting academically at-risk students has attracted much attention in the field of learning analytics. However, much of the research in this area has focused on developing advanced machine learning models to predict students’ academic performance, which alone is insufficient to improve student learning without the implementation of timely interventions. Among the studies that attempted to mitigate this limitation by deploying intervention feedback to enhance learning, few created their feedback based on established theories of effective feedback. This theoretical oversight may limit students’ uptake of the provided intervention. In response to these gaps, we conducted a study that aimed at supporting at-risk students at the early stage of an undergraduate-level course. Specifically, we developed predictive machine learning models using trace and academic data from the previous offering of a course and applied these models to identify at-risk students in the subsequent semester’s offering of the same course. For the identified at-risk students, we sent intervention emails designed by feedback experts based on a relational feedback framework designed to enhance feedback effectiveness by strengthening student-instructor relationships. We evaluated the effectiveness of the proposed approach by assessing the performance of the predictive models in terms of generalizability and measuring the impact of the feedback intervention on students’ behavioural engagement in learning. Results showed that (i) our predictive models demonstrated a high prediction accuracy (with AUC scores above 0.8) when applied to a new cohort of students; (ii) more than 30% of the identified at-risk students visited previously unengaged learning activities within two weeks following the intervention; and (iii) survey responses from 9.27% of at-risk students indicated general satisfaction with the provided feedback intervention, and 60% of the respondents expressed a preference for receiving the intervention more frequently than the twice-per-semester frequency implemented in the present study.