Attention-Based Tri-UNet for Remote Sensing Image Pan-Sharpening

Abstract

Pan-sharpening of remote sensing images is a significant method for integrating remote sensing information in the field of computer vision, where complementary and redundant information between multispectral (MS) images and panchromatic (PAN) images is used to generate high-resolution MS (HRMS) images. Inspired by the remarkable achievements of convolutional neural networks in a variety of computer-vision tasks, we incorporate domain-specific knowledge to design our attention-based triangle UNet (Tri-UNet) architecture to generate high-quality HRMS images. The attention-based Tri-UNet is mainly divided into the following three modules: 1) feature extraction; 2) feature fusion; and 3) image reconstruction. In the feature extraction step, the feature extraction module simultaneously extracts spectral and spatial information from the MS and PAN images. The feature maps are then fused in the feature fusion module, which makes the final feature image contain rich spectral and spatial information. Finally, the image reconstruction module generates a high-resolution MS image that uses the fused image as input. The attention mechanism is introduced into the image reconstruction module to make the network focus more on key information in the feature image. The experimental results demonstrate that the proposed method can generate high-quality HRMS images. A quantitative comparison and qualitative analysis of the experimental results indicate that our method is superior to the existing methods.