Automated classification of solid renal masses on contrast-enhanced computed tomography images using convolutional neural network with decision fusion

Abstract

Objectives: To develop a deep learning-based method for automated classification of renal cell carcinoma (RCC) from benign solid renal masses using contrast-enhanced computed tomography (CECT) images. Methods: This institutional review board–approved retrospective study evaluated CECT in 315 patients with 77 benign (57 oncocytomas, and 20 fat-poor angiomyolipoma) and 238 malignant (RCC: 123 clear cell, 69 papillary, and 46 chromophobe subtypes) tumors identified consecutively between 2015 and 2017. We employed a decision fusion-based model to aggregate slice level predictions determined by convolutional neural network (CNN) via a majority voting system to evaluate renal masses on CECT. The CNN-based model was trained using 7023 slices with renal masses manually extracted from CECT images of 155 patients, cropped automatically around kidneys, and augmented artificially. We also examined the fully automated approach for renal mass evaluation on CECT. Moreover, a 3D CNN was trained and tested using the same datasets and the obtained results were compared with those acquired from slice-wise algorithms. Results: For differentiation of RCC versus benign solid masses, the semi-automated majority voting-based CNN algorithm achieved accuracy, precision, and recall of 83.75%, 89.05%, and 91.73% using 160 test cases, respectively. Fully automated pipeline yielded accuracy, precision, and recall of 77.36%, 85.92%, and 87.22% on the same test cases, respectively. 3D CNN reported accuracy, precision, and recall of 79.24%, 90.32%, and 84.21% using 160 test cases, respectively. Conclusions: A semi-automated majority voting CNN-based methodology enabled accurate classification of RCC from benign neoplasms among solid renal masses on CECT. Key Points: • Our proposed semi-automated majority voting CNN-based algorithm achieved accuracy of 83.75% for the diagnosis of RCC from benign solid renal masses on CECT images. • A fully automated CNN-based methodology classified solid renal masses with moderate accuracy of 77.36% using the same test images. • Employing 3D CNN-based methodology yielded slightly lower accuracy for renal mass classification compared with the semi- automated 2D CNN-based algorithm (79.24%).