Learning to reconstruct 3D structure from object motion

Abstract

In this paper, we propose a new approach for reconstructing 3D structure from motion parallax. Instead of obtaining 3D structure from multi-view geometry or factorization, a Deep Neural Network (DNN) based method is proposed without assuming the camera model explicitly. In the proposed method, the targets are first split into connected 3D corners, and then the DNN regressor is trained to estimate the relative 3D structure of each corner from the target rotation. Finally, a temporal integration is performed to further improve the reconstruction accuracy. The effectiveness of the method is proved by a typical experiment of the Kinetic Depth Effect (KDE) in human visual system, in which the DNN regressor reconstructs the structure of a rotating 3D bent wire. The proposed method is also applied to reconstruct another two real targets. Experimental results on both synthetic and real images show that the proposed method is accurate and effective.