Ensemble learning of multiview CNN models for survival time prediction of brain tumor patients using multimodal MRI scans

Abstract

Brain tumors have been one of the most common life-threatening diseases for all mankind. There have been huge efforts dedicated to the development of medical imaging techniques and radiomics to diagnose tumor patients quickly and efficiently. One of the main aims is to ensure that preoperative overall survival time (OS) prediction is accurate. Recently, deep learning (DL) algorithms, and particularly convolutional neural networks (CNNs) achieved promising performances in almost all computer vision fields. CNNs demand large training datasets and high computational costs. However, curating large annotated medical datasets are difficult and resource-intensive. The performances of single learners are also unsatisfactory for small datasets. Thus, this study was conducted to improve the performance of CNN models on small volumetric datasets through developing a DL-based ensemble method for OS classification of brain tumor patients using multimodal magnetic resonance images (MRI). First, we proposed multiview CNNs: OS classifiers based on representing the 3D MRI data as a set of 2D slices along all three planes (axial, sagittal, and coronal) and process them using 2D CNNs. Subsequently, the predicted probabilities by the multiview CNN models were fused using standard machine learning algorithms. The proposed approach was experimentally evaluated on 163 patients obtained from the BraTS'17 training dataset. Our best model achieved an AUC and accuracy values of 0.93 and 92.9%, respectively, on classifying patients with brain tumors into two OS groups, outperforming current state-of-the-art results. In addition, the FLAIR MRI modality yielded the best classification accuracy compared to other MRI modalities. Similarly, axial projections had the best classification performance compared to coronal and sagittal projections. Our findings may provide valuable insights for physicians in advancing treatment planning via noninvasive and accurate prediction of survival using only MRIs at the time of diagnosis.