Student Engagement Recognition: Comprehensive Analysis Through EEG and Verification by Image Traits Using Deep Learning Techniques

Abstract

With the advancement in educational technologies and strategies, hybrid and online teaching methods are gaining significant popularity. Student engagement is an important factor that influences the quality of any teaching and learning process. Tracking student attention can be achieved from facial and body posture analysis. Neurophysiological signals also play a substantial role in measuring students' academic states in the learning environment. EEG signals are more sensitive to cognitive states, providing constructive perceptions of students' emotional and cognitive understandings. In this paper, we propose an engagement recognition system that detects student engagement using EEG signals by integrating levels of valence and arousal with the Russel 2D circumplex model using deep learning algorithm. The public DEAP dataset was used for training the model to predict valence and arousal values. Our system achieved an accuracy of 84.75% for detecting valence and arousal values in the four quadrants (HVHA, HVLA, LVHA, LVLA) for the DEAP dataset. For the engagement classification into four categories, 'highly engaged', 'confused', 'boredom', and 'sleepy', our system attained an accuracy of 84%. The experimental results of the proposed system were verified by analyzing the image traits of the students using the engagement indicator algorithm. Both systems were further validated through a quiz conducted at the end of the session.