Direct evidence for occlusion in stereo and motion

Abstract

Discontinuities of surface properties are the most important locations in a scene; they are crucial for segmentation because they often coincide with object boundaries. Standard approaches to discontinuity detection decouple detection of disparity discontinuities from disparity computation. We have developed techniques for locating disparity discontinuities using information internal to the stereo algorithm of [2], rather than by post-processing the stereo data. The algorithm determines displacements by maximizing the sum, at overlapping small regions, of local comparisons. The detection methods are motivated by analysis of the geometry of matching and occlusion and the fact that detection is not just a pointwise decision. Our methods can be used in combination to produce robust performance. This research is part of a project to build a “Vision Machine” [7] at MIT that integrates outputs from early vision modules. Our techniques have been extensively tested on real images.