Spatially and Temporally Segmenting Movement to Recognize Actions

Abstract

This chapter presents a Continuous Movement Recognition (CMR) framework which forms a basis for segmenting continuous human motion to recognize actions as demonstrated through the tracking and recognition of hundreds of skills from gait to twisting summersaults. A novel 3D color clone-body-model is dynamically sized and texture mapped to each person for more robust tracking of both edges and textured regions. Tracking is further stabilized by estimating the joint angles for the next frame using a forward smoothing Particle filter with the search space optimized by utilizing feedback from the CMR system. A new paradigm defines an alphabet of dynemes being small units of movement, to enable recognition of diverse actions. Using multiple Hidden Markov Models, the CMR system attempts to infer the action that could have produced the observed sequence of dynemes. 9.1 Introduction One of the biggest hurdles in human-computer interaction is the current inability for computers to recognize human activities. We introduce hierarchical Bayesian models of concurrent movement structures for temporally segmenting this complex articulated human motion. An alphabet of these motion segments are then used for recognizing activities to enable applications to extend augmented reality and novel interactions with computers. Research into computer vision based tracking and recognizing human movement has so far been mostly limited to gait or frontal posing [52]. This chapter presents a Continuous Movement Recognition (CMR) framework which forms a basis for the general analysis and recognition of continuous human motion as demonstrated through tracking and recognition of hundreds of skills from gait to twisting somersault. A novel 3D color clone-body-model is dynamically sized and texture mapped to each person for more robust tracking of both edges and textured regions. Tracking is further stabilized by estimating the joint angles for the next frame using a forward smoothing Particle filter with the search space optimized by utilizing feedback from the CMR system. A new paradigm defines an alphabet of dynemes, units of full-body 213 B. Rosenhahn et al. (eds.), Human Motion-Understanding, Modelling, Capture, and Animation, 213-241.