Computational time-resolved imaging, single-photon sensing, and non-line-of-sight imaging

Abstract

Emerging detector technologies are capable of ultrafast capture of single photons, enabling imaging at the speed of light. Not only can these detectors be used for imaging at essentially trillion frame-per-second rates, but combining them with computational algorithms has given rise to unprecedented new imaging capabilities. Computational time-resolved imaging has enabled new techniques for 3D imaging, light transport analysis, imaging around corners or behind occluders, and imaging through scattering media such as fog, murky water, or human tissue. With applications in autonomous navigation, robotic vision, human-computer interaction, and more, this is an area of rapidly growing interest. In this course, we provide an introduction to computational time-resolved imaging and single photon sensing with a focus on hardware, applications, and algorithms. We describe various types of emerging single-photon detectors, including single-photon avalanche diodes and avalanche photodiodes, which are among the most popular time-resolved detectors. Physically accurate models for these detectors are described, including modeling parameters and noise statistics used in most computational algorithms. From the application side, we discuss the use of ultrafast active illumination for 3D imaging and transient imaging, and we describe the state of the art in non-line-of-sight imaging, which requires modelling and inverting the propagation and scattering of light from a visible surface to a hidden object and back. We describe time-resolved computational algorithms used in each of these applications and offer insights on potential future directions.