Video-based performance driven facial animation

Abstract

Video-based performance driven facial animation is appealing as it offers the lowest cost, a simplified setup, and the potential use of legacy sources and uncontrolled videos. It is also difficult as it is ill-posed due to the loss of depth. This chapter introduces techniques in video-based facial reconstruction in three levels. Given the input video, the first level is to reconstruct 3D head poses and large-scale facial deformation at each frame. Representations of the facial deformation as well as the related 2D feature detection/tracking and 3D shape parameters optimization methods are introduced. Next, we discuss methods on recovering the fine-scale surface details such as emerging and disappearing wrinkles and folds. Finally, we briefly introduce the advanced applications based on the reconstructed facial performance, such as video editing and facial component enhancement.