Full Face-and-Head 3D Model with Photorealistic Texture

Abstract

In the recent period, significant progress has been achieved towards reconstructing the 3D face model from face image. With the support of the render engines and sufficient data, the reconstruction results are fine in detail. Nevertheless, the research on the 3D face reconstruction with texture from a single unrestricted face image is imperfect. The rebuild process lacks essential structure and texture information in the profile and the craniofacial region. To address this problem, we present a method of creating a 3D full face-and-head model with photorealistic texture from a single 'in-the-wild' face image in this paper. To this end, we introduce a pipeline to integrate the highly-detailed face model into the basic model. Specifically, the basic model was built by multilinear optimization, and the highly-detailed face model which represents the facial features generated by constrained illumination distribution. Additionally, to infer the invisible region texture information corresponding to the input face image, we design an effective architecture with the generative adversarial network (GAN) for panoramic UV texture generation. The final results after UV texture mapping were visualized in the experiment, which demonstrates that the model faithfully recovers the photorealistic details in arbitrary perspective. Furthermore, compared to the state-of-the-art facial modeling techniques and existing commercial solutions, our method takes less input and performs better in surface detail.