Skeleton-based continuous extrinsic calibration of multiple RGB-D kinect cameras

Abstract

Applications involving 3D scanning and reconstruction & 3D Teleimmersion provide an immersive experience by capturing a scene using multiple RGB-D cameras, such as Kinect. Prior knowledge of intrinsic calibration of each of the cameras, and extrinsic calibration between cameras, is essential to reconstruct the captured data. The intrinsic calibration for a given camera rarely ever changes, so only needs to be estimated once. However, the extrinsic calibration between cameras can change, even with a small nudge to the camera. Calibration accuracy depends on sensor noise, features used, sampling method, etc., resulting in the need for iterative calibration to achieve good calibration. In this paper, we introduce a skeleton based approach to calibrate multiple RGB-D Kinect cameras in a closed setup, automatically without any intervention, within a few seconds. The method uses only the person present in the scene to calibrate, removing the need for manually inserting, detecting and extracting other objects like plane, checker-board, sphere, etc. 3D joints of the extracted skeleton are used as correspondence points between cameras, after undergoing accuracy and orientation checks. Temporal, spatial, and motion constraints are applied during the point selection strategy. Our calibration error checking is inexpensive in terms of computational cost and time and hence is continuously run in the background. Automatic re-calibration of the cameras can be performed when the calibration error goes beyond a threshold due to any possible camera movement. Evaluations show that the method can provide fast, accurate and continuous calibration, as long as a human is moving around in the captured scene.