Dynamic Face Expression Generation with Efficient Neural Radiation Field

Abstract

Lacking of sufficient generalization ability on novel perspectives and expressions, drivable face NeRF, is still a challenging problem. In this paper, we concentrate on two aspects of the drivable face NeRF, the representation power of the driving signal and the efficiency of NeRF rendering. Firstly, we look into the utilization of world-space keypoints as the driving signal of the dynamic face. We realize this by a keypoint lifting strategy based on front keypoints to obtain stable and robust world-space keypoints, which are used to drive the deformation field and the Neural Radiance Field in the canonical space simultaneously. Second, the world-space keypoints are utilized to guide the NeRF to efficiently sample points near the face surface, and the coarse level in the original NeRF can be skipped, which significantly accelerates the rendering speed. We have verified the effectiveness and superiority of our method through good experiments.