A probabilistic framework for matching temporal trajectories: Condensation-based recognition of gestures and expressions

Abstract

The recognition of human gestures and facial expressions in image sequences is an important and challenging problem that enables a host of human-computer interaction applications. This paper describes a framework for incremental recognition of human motion that extends the “CONDENSATION” algorithm proposed by Isard and Blake (ECCV'96). Human motions are modeled as temporal trajectories of some estimated parameters over time. The CONDENSATION algorithm uses random sampling techniques to incrementally match the trajectory models to the multi-variate input data. The recognition framework is demonstrated with two examples. The first example involves an augmented office whiteboard with which a user can make simple hand gestures to grab regions of the board, print them, save them, etc. The second example illustrates the recognition of human facial expressions using the estimated parameters of a learned model of mouth motion.