Multistage Fusion with Dissimilarity Regularization for SAR/IR Target Recognition

Abstract

In this paper, we propose dissimilarity regularization with a multistage fusion stream for a synthetic aperture radar (SAR) and infrared (IR) sensor fusion using deep learning. The multistage fusion structures are composed of multiple layers for fusing all the feature maps generated by the convolutional neural networks. The proposed structure combines feature maps of equivalent levels, ensuring that the spatial information of the corresponding levels can be utilized for fusion. Dissimilarity regularization is the sum of the normalized cross-correlation between the features generated in two different single-sensor streams. The proposed regularization is added to the conventional learning problem of a single-sensor stream, and each single-sensor stream is promoted to learn the disparate types of features for fusion. To evaluate the proposed algorithm, we compare the recognition rate of the proposed algorithm with that of the conventional fusion approaches using the SAR and IR image databases. Finally, the effects of the proposed architecture and regularization on the fusion result are analyzed.