Multi-view stereo for weakly textured indoor 3D reconstruction

Abstract

A 3D reconstruction enables an effective geometric representation to support various applications. Recently, learning-based multi-view stereo (MVS) algorithms have emerged, replacing conventional hand-crafted features with convolutional neural network-encoded deep representation to reduce feature matching ambiguity, leading to a more complete scene recovery from imagery data. However, the state-of-the-art architectures are not designed for an indoor environment with abundant weakly textured or textureless objects. This paper proposes AttentionSPP-PatchmatchNet, a deep learning-based MVS algorithm designed for indoor 3D reconstruction. The algorithm integrates multi-scale feature sampling to produce global-context-aware feature maps and recalibrates the weight of essential features to tackle challenges posed by indoor environments. A new dataset designed exclusively for indoor environments is presented to verify the performance of the proposed network. Experimental results show that AttentionSPP-PatchmatchNet outperforms state-of-the-art algorithms with relative 132.87% and 163.55% improvements at the 10 and 2 mm threshold, respectively, making it suitable for accurate and complete indoor 3D reconstruction.