Real-time image registration of RGB webcams and colorless 3D time-of-flight cameras

Abstract

In line with the boom of 3D movies and cutting edge technologies, range cameras are increasingly common. Among others, time-of-flight (TOF) cameras give it the ability to capture three-dimensional images that reveal object's distances. A shortcoming of these sensors however, lies in that the majority does not provide color information (not even gray). Therefore they are useless in computer vision applications for which color is crucial. The PMD [vision] ® CamCube 3.0 is one example of an expensive colorless TOF camera. In this work, we attempt the addition of color to this camera by means of inexpensive resources. A regular webcam is stuck on top of the CamCube and its color images are registered into the TOF distance maps. To get this done, we developed an algorithm to enable real-time registration based solely on depth. Thus, this algorithm requires neither intrinsic parameters nor mono-calibration of none of the cameras. We finally show a tracking application in which a stumble is foretold if an object approaches following a threatening trajectory. © 2012 Springer-Verlag.