Multi-focus image fusion with Siamese self-attention network

Abstract

Recently, convolutional neural networks (CNNs) have achieved impressive progress in multi-focus image fusion (MFF). However, it always fails to capture sufficient discrimination features due to the local receptive field limitations of the convolutional operator, restricting most current CNN-based methods’ performance. To address this issue, by leveraging self-attention (SA) mechanism, the authors propose Siamese SA network (SSAN) for MFF. Specifically, two kinds of SA modules, position SA (PSA) and channel SA (CSA) are utilised to model the long-range dependencies across focused and defocused regions in the multi-focus image, alleviating the local receptive field limitations of convolution operators in CNN. To search a better feature representation of the input image for MFF, the captured features obtained by PSA and CSA are further merged through a learnable 1 × 1 convolution operator. The whole pipeline is in a Siamese network fashion to reduce the complexity. After training, the authors SSAN can accomplish well the fusion task with no post-processing. Experiments demonstrate that their approach outperforms other current state-of-the-art methods, not only in subjective visual perception but also in the quantitative assessment.