Direct estimation of dense scene flow and depth from a monocular sequence

Abstract

We propose a method that uses a monocular sequence for joint direct estimation of dense scene flow and relative depth, a problem that has been generally tackled in the literature with binocular or stereo image sequences. The problem is posed as the optimization of a functional containing two terms: a data term, which relates three-dimensional (3D) velocity to depth in terms of the spatiotemporal visual pattern and an L2regularization term. Based on expressing the optical flow gradient constraint in terms of scene flow velocity and depth, our formulation is analogous to the classical Horn and Schunck optical flow estimation although it involves 3D motion and depth rather than 2D image motion. The discretized Euler-Lagrange equations yield a large scale sparse system of linear equations, which we order so that the corresponding matrix is symmetric positive definite. This implies that Gauss-Seidel iterations converge, point-wise or block-wise, and afford highly efficient means of solving the equations. Examples are given to verify the scheme and its implementation.