Markov Random Fields in the Context of Stereo Vision

Abstract

The term stereo vision refers to the ability of an observer (either a human or a machine) to recover the three-dimensional information of a scene by means of (at least) two images taken from different viewpoints. Under the scope of this problem—and provided that cameras are calibrated—two subproblems are typically considered, namely, the correspondence problem, and the reconstruction problem (Trucco & Verri, 1998). The former refers to the search for points in the two images that are projections of the same physical point in space. Since the images are taken from different viewpoints, every point in the scene will project onto different image points, i.e, onto points with different coordinates in every image coordinate system. It is precisely this disparity in the location of image points that gives the information needed to reconstruct the point position in space. The second problem, i.e., the reconstruction problem, deals with calculating the disparity between a set of corresponding points in the two images to create a disparity map, and to convert this into a three-dimensional map. In this context, we will show howMarkov Random Fields (MRFs) can be effectively used. It is well known that MRFs constitute a powerful tool to incorporate spatial local interactions in a global context (Geman & Geman, 1984). So, in this chapter, we will consider local interactions that define proper MRFs to develop a model that can be applied in the process of recovery of the 3D structure of the real world using stereo pairs of images. To this end, we will briefly describe the whole stereo reconstruction process (Fig. 1), including the process of selection of features, some important aspects regarding the calibration of the camera system and related geometric transformations of the images and, finally, probabilistic analyses usable in the definition of MRFs to solve the correspondence problem. In the model to describe, both a priori and a posteriori probabilities will be separately considered and derived making use of reasonable selections of the potentials (Winkler, 1995) that define the MRFs on the basis of specific analytic models. In the next section, a general overview of a stereo system will be shown. In Sec. 3, a brief overview of some well known stereo correspondence algorithms is given. Sec. 4describes the main stages of a stereo correspondence system in which MRFS can be applied. Sec. 5describes the camera model that will be considered in this chapter together with some important related issues like: camera calibration, the epipolar constraint and image rectification. Sec. 6describes the concept of Markov random fields, and related procedures, like simulated annealing. Sec. 7 introduces MRFs for the edge detection problem. Sec. 8 describes, in detail, how MRFs can 3