OP-FCNN: an optronic fully convolutional neural network for imaging through scattering media

Abstract

Imaging through scattering media is a classical inverse issue in computational imaging. In recent years, deep learning(DL) methods have excelled in speckle reconstruction by extracting the correlation of speckle patterns. However, high-performance DL-based speckle reconstruction also costs huge hardware computation and energy consumption. Here, we develop an opto-electronic DL method with low computation complexity for imaging through scattering media. We design the "end-to-end" optronic structure for speckle reconstruction, namely optronic fully convolutional neural network (OP-FCNN). In OP-FCNN, we utilize lens groups and spatial light modulators to implement the convolution, down/up-sampling, and skip connection in optics, which significantly reduces the computational complexity by two orders of magnitude, compared with the digital CNN. Moreover, the reconfigurable and scalable structure supports the OP-FCNN to further improve imaging performance and accommodate object datasets of varying complexity. We utilize MNIST handwritten digits, EMNIST handwritten letters, fashion MNIST, and MIT-CBCL-face datasets to validate the OP-FCNN imaging performance through random diffusers. Our OP-FCNN reveals a good balance between computational complexity and imaging performance. The average imaging performance on four datasets achieves 0.84, 0.91, 0.79, and 16.3dB for JI, PCC, SSIM, and PSNR, respectively. The OP-FCNN paves the way for all-optical systems in imaging through scattering media.