Correspondence matching among stereo images with object flow and minimum spanning tree aggregation

Abstract

Stereoscopic correspondence matching is applied in many applications like robot navigation, automatic driving, virtual, and augmented reality by reconstructing the scene in three-dimensional environments. In the most real scenes, the moving objects attract more attentions than static objects and background. Thus, temporal information of consecutive frames like motion flow has been proven to improve the matching accuracy as weight prior. In this article, we propose a cost-aggregation method joining object flow and minimum spanning tree-based support region rather than aggregating on fixed size windows. However, directly combining object flow and minimum spanning tree filtering is not straightforward, due to the extremely high computing complexity. The proposed scheme implements nonlocal cost aggregation with object-based optical flow, which extends the idea of the minimum spanning tree and flow-based motion estimation to increase the matching accuracy. Temporal evidence of object flow is not only used in minimum spanning tree support region building but also incorporated with one hybrid edge prior to optimize the disparity estimation. The experimental results with synthetic stereo videos show that the proposed method outperforms other state-of-the-art stereo matching methods in most data sets. The whole stereo correspondence algorithm achieves competitive performance in terms of both accuracy and speed. We also illustrate the performance of the proposed method with the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) benchmark as one extensive comparison.