Detection of Students’ Emotions in an Online Learning Environment Using a CNN-LSTM Model †

Abstract

Emotion recognition through facial expressions is crucial in fields like healthcare, entertainment, and education, offering insights into user experiences. In online learning, traditional methods fail to capture students’ emotions effectively. This research introduces a hybrid Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) model to recognize learning emotions (interest, boredom, and confusion) during online lectures. A custom dataset was constructed by mapping action units from FER2013, CK+48, and JAFFE datasets into three learning-related categories. Images were preprocessed (grayscale conversion, resizing, normalization) and divided into training and testing sets. The CNN layers extract spatial facial features, while the LSTM layers capture temporal dependencies across video frames. Evaluation metrics included accuracy, precision, recall, and F1-score. The model achieved 98.0% accuracy, 97% precision, 98% recall, and 98% F1-score, surpassing existing CNN-only methods. This advancement enhances online learning by enabling personalized support and has applications in education, psychology, and human–computer interaction, contributing to affective computing development.