Dense depth-map estimation and geometry inference from light fields via global optimization

Abstract

Light field camera captures abundant and dense angular samplings in a single shot. The surface camera (SCam) model is an image gathering angular sample rays passing through a 3D point. By analyzing the statistics of SCam, a consistency-depth measurement is evaluated for depth estimation. However, local depth estimation still has limitations. A global method with pixel-wise plane label is presented in this paper. Plane model inference at each pixel not only recovers depth but also local geometry of scene, which is suitable for light fields with floating disparities and continuous view variation. The 2nd order surface smoothness is enforced to allow local curvature surfaces. We use a random strategy to generate candidate plane parameters and refine the plane labels to avoid falling in local minima. We cast the selection of defined labels as fusion move with sequential proposals. The proposals are elaborately constructed to satisfy sub-modular condition with 2nd order smoothness regularizer, so that the minimization can be efficiently solved by graph cuts (GC). Our method is evaluated on public light field datasets and achieves the state-of-the-art accuracy.