Joint estimation of pose and face landmark

Abstract

This paper proposes a parallel joint boosting method that simultaneously estimates poses and face landmarks. The proposed method iteratively updates the poses and face landmarks through a cascade of parallel random ferns in a forward stage-wise manner. At each stage, the pose and face landmark estimates are updated: pose probabilities are updated based on previous face landmark estimates and face landmark estimates are updated based on previous pose probabilities. Both poses and face landmarks are simultaneously estimated through sharing parallel random ferns for the pose and face landmark estimations. This paper also proposes a triangular-indexed feature that references a pixel as a linear weighted sum of three chosen landmarks. This provides robustness against variations in scale, transition, and rotation. Compared with previous boosting methods, the proposed method reduces the face landmark error by 7.1% and 12.3% in the LFW and MultiPIE datasets, respectively, while it also achieves pose estimation accuracies of 78.6% and 94.0% in these datasets.