Shortcut-upsampling block for 3d face reconstruction and dense alignment via position map

Abstract

Joint 3D face reconstruction and dense alignment based on a single image has always been a challenge in computer vision. Recently, some work achieves these two goals simultaneously by adopting the position map to represent a 3D face. In this paper, we design a novel upsampling block, denoted as shortcut-upsampling block, to construct the position map. We first utilize the residual blocks to extract and downsample the feature maps from the 2D face image, and then utilize the proposed shortcut-upsampling blocks to convert the feature maps to the corresponding position map. Shortcut-upsampling block allows our model to be only 121M and achieve excellent performance on 3D face reconstruction and dense alignment. Compared with other deep learning methods based on 3D Morphable Model(3DMM), our model is the fastest when reconstructing a 3D face. In addition, through introducing a dynamic weight loss function, our model can converge the loss to a lower value and obtain a better performance on 3D face reconstruction.