Geometric driven optical flow estimation and segmentation for 3D reconstruction

Abstract

We present a method for fully automatic 3D reconstruction from a pair of uncalibrated images in order to deal with the modeling of complex rigid scenes. A 2D triangular mesh model of the scene is calculated using a two-step algorithm mixing sparse matching and dense motion estimation approaches. The 2D mesh is iteratively refined to fit any arbitrary 3D surface. At convergence, each triangular patch corresponds to the projection of a 3D plane. The algorithm proposed here relies first on a dense disparity field. The dense field estimation modelized within a robust framework is constrained by the epipolar geometry. The resulting field is then segmented according to homographic models using iterative Delaunay triangulation. In association with a simplified self-calibration algorithm, this 2D planar model is used to obtain a VRML-compatible 3D model of the scene.