Classification of Emotions from Video Based Cardiac Pulse Estimation

Abstract

Recognizing emotion from video is an active research theme with many applications such as human-computer interaction and affective computing. The classification of emotions from facial expression is a common approach but it is sometimes difficult to differentiate genuine emotions from faked emotions. In this paper, we use a remote video based cardiac activity sensing technique to obtain physiological data to identify emotional states. We show that from the remotely sensed cardiac pulse patterns alone, emotional states can be differentiated. Specifically, we conducted an experimental study on recognizing the emotions of people watching video clips. We recorded 26 subjects that all watched the same comedy and horror video clips and then we estimated their cardiac pulse signals from the video footage. From the cardiac pulse signal alone, we were able to classify whether the subjects were watching the comedy or horror video clip. We also compare against classifying for the same task using facial action units and discuss how the two modalities compare.