Transformer-Based Regression Network for Pansharpening Remote Sensing Images

Abstract

The pansharpening entails obtaining images with uniform spectral distribution and rich spatial details by fusing multispectral images and panchromatic images, which has become a major image fusion problem in the field of remote sensing. Convolutional neural networks are widely used in image processing. We propose a transformer-based regression network (DR-NET) architecture. The first stage was feature extraction, which entailed extracting spectral information and spatial details from multispectral images and panchromatic images. The second stage was feature fusion, which entailed integrating the extracted feature information. In the third stage, image reconstruction, images with uniform distribution of spectral information, and sufficient spatial details were obtained. The fourth stage entailed optimizing the network performance and calculating the loss of shallow feature image and the image result after downsampling during image reconstruction. The performance of the DR-NET was optimized by optimizing the sum of all the loss values, which could be considered double regression. Simulated and real data experiments were conducted on the GF-2, QuickBird, and WorldView2 datasets to compare the proposed method with classical pansharpening methods. The qualitative and quantitative analyses proved that the spectral distribution of the image pansharpened using our method was uniform, the spatial details were completely retained, and the evaluation indicators were also optimal, which fully demonstrated the superior performance of the DR-NET.