Modelling Gesture

Abstract

Gesture, particularly hand gesture, is an important part of human body language which conveys messages and reveals human intention and emotional state. Automatic interpretation of gesture provides an important means for interaction and communication between human and computer, going beyond the conventional text- and graphic-based interface. Broadly speaking, human gesture can be composed of movements from any body part of a person, although the most relevant body parts are face and hand. In this sense, facial expression is a special case of gesture. Facial expression and hand movement often act together to define a coherent gesture, and can be better understood if analysed together. This is especially true when interpreting human emotional state based on visual observation of body language. Similar to facial expression, visual analysis of gesture starts with finding a suitable representation, followed by modelling and recognition. A gesture is a dynamic process, typically characterised by the spatio-temporal trajectory of body motion. It can be modelled as trajectories in a high-dimensional feature space, representing spatio-temporal correspondences between visual observations. In this context, we describe plausible methods for tracking both individual body parts and overall body movement to construct trajectories for gesture representation. Unsupervised learning is considered for automatically segmenting a continuous gesture movement sequence into atomic components and discovering the number of distinctive gesture classes. In contrast, supervised learning is studied for modelling a gesture sequence as a stochastic process, with classification of different gesture processes learned from labelled training data. We also consider the problem of affective state recognition by analysing both facial expression and body gesture together for interpreting the emotional state of a person.