HighRes-MVSNet: A Fast Multi-View Stereo Network for Dense 3D Reconstruction from High-Resolution Images

Abstract

We propose an end-to-end deep learning architecture for 3D reconstruction from high-resolution images. While many approaches focus on improving reconstruction quality alone, we primarily focus on decreasing memory requirements in order to exploit the abundant information provided by modern high-resolution cameras. Towards this end, we present HighRes-MVSNet, a convolutional neural network with a pyramid encoder-decoder structure searching for depth correspondences incrementally over a coarse-to-fine hierarchy. The first stage of our network encodes the image features to a much smaller resolution in order to significantly reduce the memory requirements. Additionally, we limit the depth search range in every hierarchy level to the vicinity of the previous prediction. In this manner, we are able to produce highly accurate 3D models while only using a fraction of the GPU memory and runtime of previous methods. Although our method is aimed at much higher resolution images, we are still able to produce state-of-the-art results on the Tanks and Temples benchmark and achieve outstanding scores on the DTU benchmark.