Enhancing Endometrial Cancer Detection by Feature Entanglement Image Generator and Multimodal Feature Learning with Attention Mechanism

Abstract

Endometrial cancer (EC) rate is rising progressively worldwide so early diagnosis of EC using medical imaging is vital to increase a patient’s survival rate. To avoid the misdiagnosis of EC on magnetic resonance imaging (MRI) scans by clinicians, an automated staging model based on deep learning has broadly emerged in medical systems. Many deep learning models like convolutional neural networks (CNNs) with transfer learning schemes are developed to classify normal and cancer patients from a sequence of MRI scans. But such models have poor sensitivity to classify EC stages from other benign lesions, resulting inaccurate diagnoses of EC. Also, the classical transfer learning schemes reduce the accuracy of detecting medical images due to the discrepancies in data distribution between the source and target domains. Hence, this article proposes a novel deep EC prediction (DeepECP) model that involves image synthesis and classification processes. First, a feature entanglement generative adversarial network (FE-GAN) is proposed for MRI synthesis that creates a desired MRI sequence according to the complementary features of multiple MRI modalities. Then, a multi-modal CNN with long short-term memory (LSTM) network followed by the fully connected (FC) layer is developed to extract a sequence of cancer features from multi-modal MRI sequences. Moreover, an attention strategy is used to fuse those extracted features and get a final feature vector, which is given to the softmax function to classify EC stages. Finally, the extensive experiments show that the DeepECP model on the TCGA-UCEC and CPTAC-UCEC datasets reaches 93.2% and 93.3% accuracy in detecting EC stages, respectively compared to the support vector machine (SVM), VGGNet-16, InceptionResNet and CNN models.