Tensor-Based Light Field Compressed Sensing and Epipolar Plane Images Reconstruction via Deep Learning

Abstract

Light field (LF) can capture the spatial and angular information of the light in one single exposure. And the LF images are widely used in various fields, especially in immersive media. The rich imaging information in the LF poses great challenges for transmission. However, LF images are sparse and redundant to some extent, which makes LF compression possible. Besides, the compressed sensing (CS) theory shows that images can be recovered from a small number of measurements when they are sparse. In this paper, we propose a Tensor-based Compressed Sensing method to compress images and Epipolar Plane Images for reconstruction (TCSEPI). This method divides the viewpoints of LF images into several regions and stacks the images in each region into a 4D tensor, which conduct CS together and yields measurements with common characteristics. Subsequently, the epipolar plane images are used to reconstruct the LF images and restore the geometric consistency information. To achieve better reconstruction results, we design two cascaded convolutional neural networks to implement the measurement matrix optimization and LF images reconstruction sequentially. Experimental results show the superior performance of TCSEPI, which achieves at least 3dB gain in PSNR and outperforms state-of-the-art in the reconstruction quality.