Hyperspectral Image Joint Super-Resolution via Local Implicit Spatial-Spectral Function Learning

Abstract

Hyperspectral image (HSI) super-resolution (SR) in both spatial and spectral dimensions is one of the most attractive research topics in HSI processing. Although recent advances in deep learning (DL) frameworks have greatly improved the performance of spatial-spectral SR reconstruction, existing methods learn discrete representations of HSI, ignoring real-world signals' continuous nature. Recently, Implicit Neural Representation (INR) has been applied to 3D surface reconstruction and image SR for continuous representation and has attracted increasing attention. In this paper, we propose the Local Implicit Spatial-spectral Function (LISSF), which learns a local continuous representation of high spatial resolution hyperspectral images (HR-HSI) from the discrete inputs. The model consists of a deep feature encoder and a spatial-spectral intensity decoder. The encoder converts the low spatial resolution multispectral image (LR-MSI) into deep features and the decoder predicts the intensity values at the given coordinates as output. Since the spatial-spectral coordinates are continuous, LISSF can achieve spatial-spectral SR in arbitrary scales, even extrapolating to higher resolutions not covered by the training data. Extensive experiments on spatial-spectral SR, spatial SR, and spectral SR demonstrate that LISSF can achieve superior performance in comparison with state-of-the-art methods. Moreover, ablation studies are performed on the effects of individual components of LISSF.