Infrared and visible image fusion network based on multistage progressive injection

Abstract

Currently, single-sensor data is frequently utilized in technologies such as object detection. However, in certain scenarios, some sensors may experience failure or information loss, significantly impacting model performance. Given the notable complementarity between infrared and visible images at the information level, it is helpful to improve the robustness and reliability of the model by fusing them and applying them to object detection and other technologies. Nevertheless, most prevalent infrared and visible image fusion methods focus on exploring invariant features across multimodal images, somewhat neglecting the inherent characteristics of the images themselves, leading to issues like structural blurriness and unclear detailed textures in the fused images, which fail to meet application demands. To overcome this challenge, this paper proposes an Infrared and visible image fusion network based on multistage progressive injection, termed MPIFusion. To effectively leverage the inherent characteristics and complementarity of images, and address the issues of structural blurriness and unclear detailed textures in fused images, we introduce a Dual-channel Shallow Detail Fusion Module (DC-SDFM) and a Deep Feature Fusion Block (DFFB). These modules first enhance the original features and then fuse the hierarchical features of infrared and visible images with the aid of an attention mechanism module. Furthermore, we construct a progressive injection layer based on the Information Fusion Module (IFM), integrating the fused features within the same framework to generate high-quality infrared and visible fused images. Extensive experiments demonstrate that our MPIFusion outperforms 15 existing fusion methods in terms of performance. The generated fused images not only highlight global and local detail features but also exhibit higher clarity and contrast. Finally, we apply fusion methods to object detection scenarios, and the results show that MPIFusion exhibits significant superiority in such scenarios, providing more robust and reliable image support. The source code is available at https://github.com/Kaixuan-Chang/MPIFusion.