A Comparative Approach for Facial Expression Recognition in Higher Education Using Hybrid-Deep Learning from Students' Facial Images

Abstract

Online education has become increasingly common due to the Covid-19 pandemic. A key difference between online and face-to-face education is that instructors often cannot see their students' facial expressions online. This is problematic because facial expressions can help an instructor gauge engagement and understanding. Therefore, it would be useful to find a mechanism whereby students' facial expressions during an online lecture could be monitored. This information can be used as feedback for teachers to change a particular teaching method or maintain another method. This research presents a system that can automatically distinguish students' facial expressions. These comprise eight expressions (anger, attention, disgust, fear, happiness, neutrality, sadness, and surprise). The data for this research was collected from pictures of 70 university students' facial expressions. The data included 6720 images of students' faces distributed equally among the eight expressions mentioned above, that is, 840 images for each category. In this paper, pre-trained deep learning networks (AlexNet, MobileNetV2, GoogleNet, ResNet18, ResNet50, and VGG16) with transfer learning (TL) and K-fold validation (KFCV) were used for recognizing the facial expressions of students. The experiments were conducted using MATLAB 2021a and the best results were recorded by ResNet18 for F1-score and for AUC curve 99%, and 100% respectively.