Deep learning for computational imaging: from data-driven to physics-enhanced approaches

Abstract

Computational imaging (CI) leverages the joint optimization of optical system design and reconstruction algorithms, enabling superior performance in terms of dimensionality, resolution, efficiency, and hardware complexity. It has found widespread applications in medical diagnosis and astronomy, among others. Recently, deep learning (DL) has changed the paradigm of CI by harnessing learned priors from data through trained neural network models. However, widely used data-driven DL-based CI methods encounter difficulties related to training data acquisition, computation requirements, generalization, and interpretability. Recent studies have indicated that integrating the physics prior of the CI system into various components of DL pipelines (including training data, network design, and loss functions) holds promise for alleviating these challenges. To provide readers with a better understanding of the current research status and ideas, we present an overview of the state-of-the-art in DL-based CI. We begin by briefly introducing the concepts of CI and DL, followed by a comprehensive review of how DL addresses inverse problems in CI. Particularly, we focus on the emerging physics-enhanced approaches. We highlight the perspectives of future research directions and the transfer to real-world applications.