Using 3-dimensional meshes to combine image-based and geometry-based constraints

Abstract

To recover complicated surfaces, single information sources often prove insufficient. In this paper, we present a unified framework for 3-D shape reconstruction that allows us to combine image-based constraints, such as those deriving from stereo and shape-from-shading, with geometry-based ones, provided here in the form of 3-D points, 3-D features or 2-D silhouettes. Our approach to shape recovery is to deform a generic object-centered 3-D representation of the surface so as to minimize an objective function. This objective function is a weighted sum of the contributions of the various information sources. We describe these various terms individually, our weighting scheme and our optimization method. Finally, we present results on a number of difficult images of real scenes for which a single source of information would have proved insufficient.