Robust multi-view face alignment based on cascaded 2D/3D face shape regression

Abstract

In this paper, we present a cascaded regression algorithm for multi-view face alignment. Our method employs a two-stage cascaded regression framework and estimates 2D and 3D facial feature points simultaneously. In stage one, 2D and 3D facial feature points are roughly detected on the input face image, and head pose analysis is applied based on the 3D facial feature points to estimate its head pose. The face is then classified into one of three categories, namely left profile faces, frontal faces and right profile faces, according to its pose. In stage two, accurate facial feature points are detected by using an appropriate regression model corresponding to the pose category of the input face. Compared with existing face alignment methods, our proposed method can better deal with arbitrary view facial images whose yaw angles range from −90 to 90°. Moreover, in order to enhance its robustness to facial bounding box variations, we randomly generate multiple bounding boxes according to the statistical distributions of bounding boxes and use them for initialization during training. Extensive experiments on public databases prove the superiority of our proposed method over state-of-the-art methods, especially in aligning large off-angle faces.