A Design and Implementation of an Online Video Lecture System based on Facial Expression Recognition

Abstract

Real-time online education facilitates education without requiring instructors and students to be physically present in the exact location, allowing for interactions similar to face-to-face teaching. However, when many students participate simultaneously, instructors face challenges, such as spending a lot of time understanding each student's learning status. Mainly, systems utilized for real-time online education, initially developed for business meetings, have limitations when repurposed for educational uses. Therefore, this paper presents the design and implementation of a video lecture system for real-time online education based on facial expression recognition. The system is a video lecture system based on Facial Expression Recognition, implemented with a web browser method, a classroom tool platform supporting teacher-student interaction, and facial expression recognition features based on an artificial intelligence engine. Engagement metrics utilize fundamental values like facial recognition, total learning time, and video playback to measure student engagement through facial expressions. Additionally, structured numerical data such as eye-tracking, motion tracking, drowsiness tracking, speaking, effective chatting, hand-raising, polling, and screen sharing are weighted and aggregated for calculation. Weights can be adjusted according to the nature of the lecture, such as discussion-based learning or cooperative learning. The system calculates student engagement levels, categorizing them as ‘Active (90 points)', ‘Moderate (80 points)', and ‘Insufficient (70 points)', and is designed to provide this data to instructors asynchronously. Furthermore, a focus group interview was conducted with Edtech experts to validate the implemented system, and positive responses were obtained.