3D deformable spatial pyramid for dense 3D motion flow of deformable object

Abstract

This paper presents an algorithm for finding the dense motion flow of deformable objects from RGB-D images. We introduce a 3D deformable spatial pyramid model by reformulating the previous 2D deformable spatial pyramid model [1] with depth information. Our algorithm recasts the problem of estimating 3D motion of deformable objects as a problem of estimating 2D motions of a set of grid cells where each pixel contains a viewpoint-invariant feature vector. These grid cells are controlled by a pyramid graph model. Our approach significantly reduces the computational cost through a 2D correspondence search and efficiently handles even large deformations with the pyramid graph model. As demonstrated in the experimental results, the proposed algorithm shows robustness in various deformation scenarios.