Disparity Estimation Using Convolutional Neural Networks with Multi-scale Correlation

Abstract

Disparity estimation is a long-standing task in computer vision and multiple approaches have been proposed to solve this problem. A recent work based on convolutional neural networks, which uses a correlation layer to perform the matching process, has achieved state-of-the-art results for the disparity estimation task. This correlation layer employs a single kernel unit which is not suitable for low texture content and repeated patterns. In this paper we tackle this problem by using a multi-scale correlation layer with several correlation kernels and different scales. The major target is to integrate the information of the local matching process by combining the benefits of using both a small correlating scale for fine details and bigger scales for larger areas. Furthermore, we investigate the training approach using horizontally elongated patches that fits the disparity estimation task. The results obtained demonstrate the benefits of the proposed approach on both synthetic and real images.