Msnet: A multi-stream fusion network for remote sensing spatiotemporal fusion based on transformer and convolution

37Citations
Citations of this article
23Readers
Mendeley users who have this article in their library.

Abstract

Remote sensing products with high temporal and spatial resolution can be hardly obtained under the constrains of existing technology and cost. Therefore, the spatiotemporal fusion of remote sensing images has attracted considerable attention. Spatiotemporal fusion algorithms based on deep learning have gradually developed, but they also face some problems. For example, the amount of data affects the model’s ability to learn, and the robustness of the model is not high. The features extracted through the convolution operation alone are insufficient, and the complex fusion method also introduces noise. To solve these problems, we propose a multi-stream fusion network for remote sensing spatiotemporal fusion based on Transformer and convolution, called MSNet. We introduce the structure of the Transformer, which aims to learn the global temporal correlation of the image. At the same time, we also use a convolutional neural network to establish the relationship between input and output and to extract features. Finally, we adopt the fusion method of average weighting to avoid using complicated methods to introduce noise. To test the robustness of MSNet, we conducted experiments on three datasets and compared them with four representative spatio-temporal fusion algorithms to prove the superiority of MSNet (Spectral Angle Mapper (SAM) < 0.193 on the CIA dataset, erreur relative global adimensionnelle de synthese (ERGAS) < 1.687 on the LGC dataset, and root mean square error (RMSE) < 0.001 on the AHB dataset).

References Powered by Scopus

Deep residual learning for image recognition

174322Citations
N/AReaders
Get full text

Image quality assessment: From error visibility to structural similarity

44825Citations
N/AReaders
Get full text

Densely connected convolutional networks

28581Citations
N/AReaders
Get full text

Cited by Powered by Scopus

TransUNetCD: A Hybrid Transformer Network for Change Detection in Optical Remote-Sensing Images

230Citations
N/AReaders
Get full text

Trs: Transformers for remote sensing scene classification

105Citations
N/AReaders
Get full text

SwinSTFM: Remote Sensing Spatiotemporal Fusion Using Swin Transformer

46Citations
N/AReaders
Get full text

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Cite

CITATION STYLE

APA

Li, W., Cao, D., Peng, Y., & Yang, C. (2021). Msnet: A multi-stream fusion network for remote sensing spatiotemporal fusion based on transformer and convolution. Remote Sensing, 13(18). https://doi.org/10.3390/rs13183724

Readers over time

‘21‘22‘23‘24‘25036912

Readers' Seniority

Tooltip

PhD / Post grad / Masters / Doc 7

78%

Professor / Associate Prof. 1

11%

Lecturer / Post doc 1

11%

Readers' Discipline

Tooltip

Computer Science 5

56%

Environmental Science 2

22%

Engineering 1

11%

Agricultural and Biological Sciences 1

11%

Save time finding and organizing research with Mendeley

Sign up for free
0