A fully automatic approach to facial feature detection and tracking

Abstract

The detection of facial features is a necessary step for a wide range of applications e.g. person verification, lipreading and model-based face coding. Due to changes in illumination, visual angle and facial expressions, the variability of facial features in their appearance is high. A robust approach to facial feature detection has to handle these variations. In this framework, we present an approach for the extraction of eyebrows, eyes, nostrils, mouth and chin. Our approach for facial feature extraction is based on the observation that facial features differ from the rest of the face because of their low brightness. Thus, we detect facial feature candidates by evaluating the topographic greylevel relief of the face region. Based on vertical symmetry, distances between facial features and the assessment of each facial feature, we choose the best face constellation. Incomplete face constellations are considered as well. Once facial features are detected in an image sequence, they can be tracked over time. We perform facial feature tracking by block matching. The best-match position is refined by minima analysis. The success of our approach was tested on 38 different image sequences containing faces.