Super-resolution of depth map exploiting planar surfaces

Abstract

Depth map, with per-pixel depth values, represents the relative distance between object in the scene and the capturing depth camera. Hence, it has been widely used in 3D applications and Depth Image-Based Rendering (DIBR) technique to provide an immersive 3D and free-viewpoint experience to the viewers. Depth maps could be generated by using software- or hardware-driven techniques. However, most generated depth maps suffer from a combination of the following shortcomings: noise, holes and limited spatial resolution. Therefore, to tackle the limited spatial resolution problem of Time-of-Flight depth images, in this paper, we present a planar-surface-based depth map super-resolution approach, which interpolates depth images by exploiting the equation of each detected planar surface. Aided with these equations the surfaces will be categorized into three groups, namely: planar surfaces, non-planar surfaces, and finally edges. For the first category the analytical equations of the planar surfaces will be used to super-resolve them, while a traditional interpolation method will be used for the non-planar surfaces, whereas, a combination of the two previous approaches will be used to up-sample edges. Both quantitative and qualitative experimental results demonstrate the effectiveness and robustness of our approach over the benchmark methods.