Robust segment-based stereo using cost aggregation

Abstract

A general approach in segment-based stereo methods is to segment an image and estimate a 3-D plane for each segment, or group of segments. Inherently, such methods are sensitive to segmentation parameters and intolerant to segmentation errors. We propose a novel algorithm for generating sub-pixel accurate disparities on a per-pixel basis, thus alleviating the problems arising from methods that estimate disparities on a per-segment basis. An initial disparity map, generated using any fast local method, is used in conjunction with a color-based segmentation map to generate a set of planes. The cost of assigning each plane to every pixel is computed, and the powerful spanning-tree based cost aggregation approach is used to assign a plane label to each pixel. The steps of plane estimation and assignment are repeated in an iterative framework to enhance the results. An evaluation on the Middlebury database demonstrates the robustness of our method to segmentation parameters and disparity initialization. We also show that the disparity map accuracy for the proposed method compares favorably with most state-of-the-art approaches, being ranked 1st in average percentage of bad pixels, and 11th overall.