Monocular surface reconstruction using 3D deformable part models

Abstract

Our goal in this work is to recover an estimate of an object’s surface from a single image. We address this severely ill-posed problem by employing a discriminatively-trained graphical model: we incorporate prior information about the 3D shape of an object category in terms of pairwise terms among parts, while using powerful CNN features to construct unary terms that dictate the part placement in the image. Our contributions are three-fold: firstly, we extend the Deformable Part Model (DPM) paradigm to operate in a three-dimensional pose space that encodes the putative real-world coordinates of object parts. Secondly, we use branch-and-bound to perform efficient inference with DPMs, resulting in accelerations by two orders of magnitude over lineartime algorithms. Thirdly, we use Structured SVM training to properly penalize deviations between the model predictions and the 3D ground truth information during learning. Our inference requires a fraction of a second at test time and our results outperform those published recently in [17] on the PASCAL 3D+ dataset.