Facial expression recognition for learning status analysis

Abstract

Facial expression provides an important clue for teachers to know the learning status of students. Thus, vision-based expression analysis is valuable not only in Human-Computer Interface but also in e-Learning. We propose a computer vision system to automatically analyze learners' video to recognize nonverbal facial expressions to discover learning status of students in distance education. In the first stage, Adaboost classifiers are applied to extract candidates of facial parts. Then spatial relationships are utilized to determine the best combination of facial features to form a feature vector. In the second stage, each feature vector sequence is trained and recognized as a specific emotional expression using Hidden Markov Model (HMM). The estimated probabilities of six expressions are combined into an expression vector. The last stage is to analyze the expression vector sequence to figure out the learning situation of the student. Gaussian Mixture Model (GMM) is applied to evaluate three learning scores (Understanding, Interaction, and Consciousness) that are integrated into a status vector. Each evaluated status vector reflects the learning status of a student and is helpful to not only teachers but also students for improving teaching and learning. © 2011 Springer-Verlag.