Edge-guided single facial depth map super-resolution using cnn

Abstract

In recent years, consumer depth cameras have been widely used in digital entertainment and human-machine interaction due to the advantages of real-time performance and low cost. Facial depth maps have shown great potential in 3D-face-related studies. However, disadvantages of low resolution and precision limit its further applications. In this work, the authors propose an edge-guided convolutional neural network for single facial depth map super-resolution. It consists of two parts: an edge prediction sub-network and a depth reconstruction sub-network. The edge prediction sub-network generates an edge guidance map to guide the depth reconstruction sub-network to recover sharp edges and fine structures. Effective data augmentation methods are proposed as well. The network is patch-based and able to cope with any size of the input depth maps. In addition, it is insensitive to the face pose since the synthetic training dataset they generated covers a wide range of face poses. The proposed method is validated with three datasets including a synthetic facial depth data set, a real Kinect V2 facial depth data set and Middlebury Stereo Data set. Experimental results show that it outperforms the state-of-the-art methods on all the three data sets.