Dynamic Projection Mapping Using Kinect-Based Skeleton Tracking

Abstract

This paper presents a novel dynamic projection mapping which tracks a skeleton of a user and projects a sequence of images on the body of the user based on robust camera-project calibration without depending on accurate camera calibration using RGB-D image of the Kinect. As the first step of the proposed method, the camera-projector calibration using RGB-D images of the Kinect is performed without the printed calibration board. In the second step, the planes for the torso and palms are defined in 3D world coordinates based on the 2D skeleton information and RGB-D image. In the last step, the projection screen coordinates for the planes of the torso and palms are computed based on the homography between the camera image and the screen image for the projection with considering depth values of the planes of torso and palms. We proved the accuracy and convenience of the proposed method through the projection mapping experiments on a user in various poses.