Visual Geometry Group-SwishNet-Based Asymmetric Facial Emotion Recognition for Multi-Face Engagement Detection in Online Learning Environments

Abstract

In the contemporary global educational environment, the automatic assessment of students’ online engagement has garnered widespread attention. A substantial number of studies have demonstrated that facial expressions are a crucial indicator for measuring engagement. However, due to the asymmetry inherent in facial expressions and the varying degrees of deviation of students’ faces from a camera, significant challenges have been posed to accurate emotion recognition in the online learning environment. To address these challenges, this work proposes a novel VGG-SwishNet model, which is based on the VGG-16 model and aims to enhance the recognition ability of asymmetric facial expressions, thereby improving the reliability of student engagement assessment in online education. The Swish activation function is introduced into the model due to its smoothness and self-gating mechanism. Its smoothness aids in stabilizing gradient updates during backpropagation and facilitates better handling of minor variations in input data. This enables the model to more effectively capture subtle differences and asymmetric variations in facial expressions. Additionally, the self-gating mechanism allows the function to automatically adjust its degree of nonlinearity. This helps the model to learn more effective asymmetric feature representations and mitigates the vanishing gradient problem to some extent. Subsequently, this model was applied to the assessment of engagement and provided a visualization of the results. In terms of performance, the proposed method achieved high recognition accuracy on the JAFFE, KDEF, and CK+ datasets. Specifically, under 80–20% and 10-fold cross-validation (CV) scenarios, the recognition accuracy exceeded 95%. According to the obtained results, the proposed approach demonstrates higher accuracy and robust stability.