Deep learning-based fusion of CT-MRI modalities in medical imaging

Abstract

Aiming at the problems of important feature loss, unprominent detail performance and unclear texture in multimodal medical image fusion, a method for the fusion of computed tomography (CT) images with magnetic resonance imaging (MRI) images by using Generative Adversarial Networks (GAN) and Wavelet Transform is proposed. The generator is for the highfrequency feature image, and the double discriminator is for the fused image after inverse transformation. The high frequency feature image is fused by the GAN model, and the low frequency feature image is fused by Wavelet Transform. Experimental results show that compared with the current advanced fusion algorithms, the proposed method has richer texture details in subjective performance, and the contour edge information is clearer and more prominent. The fused image can be effectively applied to medical diagnosis and further improve the diagnosis efficiency.