Progressive non-rigid registration of temporal mesh sequences

Abstract

We present an algorithm for progressive mesh registration to provide temporal consistency, simplify temporal texture editing and optimize the data rate of 3D mesh sequences recorded in a volumetric capture studio. The capture pipeline produces a sequence of manifold meshes with varying connectivity. We split the sequence into groups of frames that will share connectivity and chose a keyframe in each group that is progressively deformed to approximate the surface of the adjacent meshes. Our algorithm works with a coarse-to-fine ICP approach, which makes it robust against large deformations in the scene while preserving small details. Applying a deformation graph constrains the transformations to be locally as-rigid-as-possible, while allowing it to work with any natural objects in the scene, not just humans. We show how to robustly track sequences of human actors with varying clothing over hundreds of frames recorded in a volumetric capture studio. We verify our results with a publicly available dataset of more than 40,000 frames. Our mesh registration takes less than five seconds per frame on a single desktop machine and has been successfully integrated into a volumetric capture pipeline for commercial use.