Reflective regression of 2d-3d face shape across large pose

Abstract

In this paper we present a novel reflective method to estimate 2D-3D face shape across large pose. We include the knowledge that a face is a 3D object into the learning pipeline, and formulate face alignment as a 3DMM fitting problem, where the camera projection matrix and 3D shape parameters are learned by an extended cascaded pose regression framework. In order to improve algorithm robustness in difficult poses, we introduce a reflective invariant metric for failure alert. We investigate the relation between reflective variance and face misalignment error, and find there is strong correlation between them. Consequently this finding is exploited to provide feedback to our algorithm. For the samples predicted as failure, we restart the algorithm with better initialisations based on explicit head pose estimation, which enhances the possibility of convergence. Extensive experiments on the challenging AFLW and AFW datasets demonstrate that our approach achieves superior performance over the state-of-the-art methods.