An Exploration of Facial Expression Tracking in Affective HCI

Abstract

We report an investigation exploring facial expression tracking in the evaluation of software usability and content. Participants’ faces were videoed whilst taking a Web-based quiz. The quiz contained two relatively ordinary HCI events as stimuli: a surprise alert (the stronger stimulus intended to evoke surprise) and questions with high affective content (the weaker stimulus intended to evoke amusement). Physiological arousal was also recorded. The videos for the periods around each event were analysed by commercially available facial movement tracking software, and the output used to drive a 3D virtual head. The movements of the real and virtual faces were assessed by human judges. Users’ real faces were judged to have responded to both stimuli, but more to the stronger stimulus than the weaker. Facial response did not always concur with physiological arousal. Judgements of the virtual faces indicated that the tracker successfully detected responses to the stronger stimulus but had mixed success with the weaker stimulus. The track data appeared to distinguish between amusement and surprise. These results suggest that facial expression tracking can be a viable and useful technique for affective computing. We suggest that the intentional and communicative use of facial expression may be a fruitful domain for future applications.