Sparse- And noisy-to-dense depth map upsampling based on mesh and colour consistency

Abstract

The paper presents a mesh- and colour-consistency-based depth map upsampling method from sparse depth information with various sampling structures under noisy conditions. In addition, we applied the proposed method to generate spatially consistent depth maps and a dense 3D point cloud from a sparse and noisy initial 3D point cloud. In the proposed method, triangulation is first performed on an image plane, whose sparse depth information is contaminated by noise and have irregular sampling structures. Then, an iterative discontinuity-preserving noise reduction process is enforced in the triangulation. After the noise reduction, a depth assignment method based on colour consistency and triangulation is used to generate a dense depth map. The experiment results show that the proposed method can provide a more accurate depth map than previous sparse-to-dense depth map upsampling methods. Furthermore, the application results verify the applicability and potential of the proposed method to various areas with inherent sparsity and irregularity of the input depth information, such as multi-view stereo.