Real-time facial feature localization by combining space displacement neural networks

Abstract

We present in this paper a new facial feature localizer. It uses a kind of auto-associative neural network trained to localize specific facial features (like eyes and mouth corners) in orientation-free face-images (i.e. images where faces are rotated in-plane and out-of-plane). To increase localization accuracy, two extensions are presented. The first one uses space displacement neural networks instead of classical, fully-connected networks. The second one combines several specialized networks trained to deal with each face orientation. A gating network is then used for combination. Finally, a two stage localizer is presented, which increases speed. Thorough evaluation is performed; including sensitivity to identity, noise and occlusions. The mean localization error (estimated on more than 4000 test images) is about 15% and the system can perform 40 images/s. © 2007 Elsevier B.V. All rights reserved.