Human body shape and motion tracking by hierarchical weighted ICP

Abstract

We present a new approach for tracking both the human body shape and the whole body motion with complete six DOF of each body limb without imposing rotation or translation constraints. First, a surface mesh with highly improved quality is obtained by using our new silhouette-based visual hull reconstruction method for each frame of multi-view videos. Then, a skinned mesh model is fitted to the data using hierarchical weighted ICP (HWICP) algorithm, where an easy-to-adjust strategy for selecting the set of ICP registration points is given based on the weights of the skinned model and the Approximate Nearest Neighbors (ANN) method is applied for fast searching nearest neighbors. By comparing HWICP with the general hierarchical ICP (Iterative Closest Point) method based on synthetic data, we demonstrate the power of weighting corresponding point pairs in HWICP, especially when adjacent body segments of target are near 'cylindrical-shaped'. © 2011 Springer-Verlag.