Stitching of depth and color images from multiple RGB-D sensors for extended field of view

Abstract

Existing commodity RGB-D sensors have a limited field of view compared with a nearly 360° horizontal field of view of Lidar. This article presents a method to extend the field of view by stitching depth and color images from multiple RGB-D sensors to form a depth and a color panorama. Firstly, a set of 3-D matched key points are constructed based on 2-D matched key points and corresponding depth values to estimate transformation matrices between multiple RGB-D sensors. Then, depth and color images are projected to the 3-D space using estimated transformation matrices to construct textured 3-D points. Finally, projecting textured 3-D points to a compositing surface can create the depth panorama and the color panorama. Experiments validated the accuracy and efficiency of the proposed method.