We propose a light-weight yet highly robust method for real-time human performance capture based on a single depth camera and sparse inertial measurement units (IMUs). Our method combines non-rigid surface tracking and volumetric fusion to simultaneously reconstruct challenging motions, detailed geometries and the inner human body of a clothed subject. The proposed hybrid motion tracking algorithm and efficient per-frame sensor calibration technique enable non-rigid surface reconstruction for fast motions and challenging poses with severe occlusions. Significant fusion artifacts are reduced using a new confidence measurement for our adaptive TSDF-based fusion. The above contributions are mutually beneficial in our reconstruction system, which enable practical human performance capture that is real-time, robust, low-cost and easy to deploy. Experiments show that extremely challenging performances and loop closure problems can be handled successfully.
CITATION STYLE
Zheng, Z., Yu, T., Li, H., Guo, K., Dai, Q., Fang, L., & Liu, Y. (2018). HybridFusion: Real-time performance capture using a single depth sensor and sparse IMUs. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 11213 LNCS, pp. 389–406). Springer Verlag. https://doi.org/10.1007/978-3-030-01240-3_24
Mendeley helps you to discover research relevant for your work.