Constrained optimization for plane-based stereo

Abstract

Depth and surface normal estimation are crucial components in understanding 3D scene geometry from calibrated stereo images. In this paper, we propose visibility and disparity magnitude constraints for slanted patches in the scene. These constraints can be used to associate geometrically feasible planes with each point in the disparity space. The new constraints are validated in the PatchMatch Stereo framework. We use these new constraints not only for initialization, but also in the local plane refinement step of this iterative algorithm. The proposed constraints increase the probability of estimating correct plane parameters, and lead to an improved 3D reconstruction of the scene. Furthermore, the proposed constrained initialization reduces the number of iterations before convergence to the optimal plane parameters. In addition, as most stereo image pairs are not perfectly rectified, we modify the view propagation process by assigning the plane parameters to the neighbors of the candidate pixel. To update the plane parameters in the plane refinement step, we use a gradient free non-linear optimizer. The benefits of the new initialization, propagation, and refinement schemes are demonstrated.