Face2Face ρ : Real-Time High-Resolution One-Shot Face Reenactment

Abstract

Existing one-shot face reenactment methods either present obvious artifacts in large pose transformations, or cannot well-preserve the identity information in the source images, or fail to meet the requirements of real-time applications due to the intensive amount of computation involved. In this paper, we introduce Face2Face ρ, the first Real-time High-resolution and One-shot (RHO, ρ ) face reenactment framework. To achieve this goal, we designed a new 3DMM-assisted warping-based face reenactment architecture which consists of two fast and efficient sub-networks, i.e., a u-shaped rendering network to reenact faces driven by head poses and facial motion fields, and a hierarchical coarse-to-fine motion network to predict facial motion fields guided by different scales of landmark images. Compared with existing state-of-the-art works, Face2Face ρ can produce results of equal or better visual quality, yet with significantly less time and memory overhead. We also demonstrate that Face2Face ρ can achieve real-time performance for face images of 1440 × 1440 resolution with a desktop GPU and 256 × 256 resolution with a mobile CPU.