A Clustering Method to Detect Disengaged Students from Their Code Submission History

Abstract

Computer Science educators benefit from knowing which of their students need help or feel the material is difficult. Collecting this information is, typically, done via surveys. However, surveying students is time-consuming and not every student answers. Additionally, with surveys, teachers identify struggling students only after-the-fact, when nothing can be done to help them. This paper proposes a machine learning approach to cluster students into groups with similar features; the weak-performing learners, who consider the material and/or programming difficult, the average-performing learners, who are learning the material normally, and the best-performing learners, who consider programming to be an easy task and do not need additional assistance. The clustering is data-driven as it is based on the collection of code snapshots (i.e., submissions). The clusters are then formed from a transition probability matrix, computed using the submission history of the student, and a state machine. We demonstrate the effectiveness of the clustering approach to create clusters with significant differences in perceived student difficulty by cross-validating the formed clusters with survey data collected from a CS1 course. The clusters also contain differences in programming behavior. The proposed method can be applied to the classroom setting to identify students benefitting from assistance during the course in real-time, without surveys.