Markerless motion capture with multi-view structured light

Abstract

We present a multi-view structured light system for markerless motion capture of human subjects. In contrast to existing approaches that use multiple camera streams or single depth sensors, we reconstruct the scene by combining six partial 3D scans generated from three structured light stations surrounding the subject. We avoid interference between multiple projectors through time multiplexing and synchronization across all cameras and projectors. We generate point clouds from each station, convert them to partial surfaces, and merge them into a single coordinate frame. We develop algorithms to reconstruct dynamic geometry using a template generated by the system itself. Specifically, we deform the template to each frame of the captured geometry by iteratively aligning each bone of the rigged template. We show the effectiveness of our system for a 50-second sequence of a moving human subject.