Machine learning approach for student engagement automatic recognition from facial expressions

Abstract

Digital revolution can significantly improve the quality of education. There have been already discussions for a long time about the advantages, disadvantages and opportunities for transforming traditional classroom activities. Modern students use smartphones and tablets "from birth", and for the majority of academic subject areas students can often obtain more complete, accurate and up-to-date information from the Internet than from lectures. Is it interesting for students to learn? Are they in time with the professor? Is the presentation clear? How deep are students engaged in learning in the classroom? These issues come to the forefront in the era of digital education. However, it was almost unrealistic to control the level of student engagement until recently: for example, only in the Moscow campuses of the Financial University the classes are held daily from 8.30 to 22.00 in more than 500 classrooms. Existing information systems for student engagement automatic recognition are focused on analyzing individual engagement of students and schoolchildren. We propose a system that constantly analyzes the flow of data from video cameras installed in classrooms, uses machine learning models to identify students' faces, recognize their emotions and determine the level of engagement, and then aggregates engagement data on student groups, faculties, courses, etc. on interactive dashboards. The training dataset consisted of 2,000 faces was used for machine learning model identification with boosted decision trees algorithm (ADABoost). The quality metrics (Accuracy, Precision, Recall, AUC) on a test dataset of 500 students faces were all above 0,81. The system is developed as an elastically scalable cloud service that automatically collects video streams from cameras installed in classrooms and forms the resulting metrics of the students and groups' engagement in the Microsoft Azure cloud.