A learning-based high-level human computer interface for face modeling and animation

Abstract

This paper describes a system for animation and modeling of faces in images or in 3D. It provides high-level control of facial appearance to users, due to a learning-based approach that extracts class-specific information from a database of 3D scans. The modification tools include changes of facial attributes, such as body weight, masculine or feminine look, or overall head shape. Facial expressions are learned from examples and can be applied to new individuals. The system is intrinsically based on 3D face shapes and surface colors, but it can be applied to existing images as well, using a 3D shape reconstruction algorithm that operates on single images. After reconstruction, faces can be modified and drawn back into the original image, so the users can manipulate, animate and exchange faces in images at any given pose and illumination. The system can be used to create face models or images from a vague description or mental image, for example based on the recollection of eyewitnesses in forensic applications. For this specific problem, we present a software tool and a user study with a forensic artist. Our model-based approach may be considered a prototype implementation of a high-level user interface to control meaningful attributes in human faces. © 2007 Springer-Verlag Berlin Heidelberg.