MH-Net: A Learnable Multi-Hypothesis Network for Compressed Video Sensing

Abstract

This paper establishes a novel recovery network called MH-Net - a framework for compressed video sensing (CVS) based on recently emerging deep neural networks (DNNs) techniques. MH-Net exploits temporal correlation between frames in the form of multi-hypothesis (MH) prediction, and learns a high-dimensional domain which is more robust for prediction generation. After the MH prediction, a special residual network is used in MH-Net to reconstruct the residuals between the MH prediction and the desired frame from their measurements. The final reconstruction is derived by adding the reconstructed residuals to the MH prediction. Unlike the block-wise reconstruction in existing DNN-based CS architecture, MH-Net builds a mapping from block measurements to a complete frame reconstruction, leading to better reconstruction quality. Benefitting from the DNN's nature, the forward propagation of MH-Net is extremely fast, making it suitable for real-time applications. Experimental results show that MH-Net presents a better recovery performance compared with existing DNN-based recovery methods and traditional iterative recovery algorithms.