Residual deep learning system for mass segmentation and classification in mammography

Abstract

Automatic extraction of breast mass in mammogram (MG) images is a challenging task due to the varying sizes, shapes, and textures of masses. Moreover, the density of MGs makes mass detection very challenging since masses can be hidden in dense MGs. In this paper, we propose a residual deep learning (DL) system for mass segmentation and classification in mammography. The overall proposed system consists of two cascaded parts: 1) a residual attention U-Net model (RU-Net) to precisely segment mass lesions in MG images, followed by 2) a ResNet classifier to classify the detected binary segmented lesions into benign or malignant. The proposed semantic based CNN model, RU-Net, has the basic architecture of the U-Net model, which extracts contextual information combining low-level feature with high-level ones. We have modified the U-Net structure by adding residual attention modules in order to preserve the spatial and context information, help the network have deeper architecture, and handles the gradient vanishing problem. We compared the performance of the proposed RU-Net model with those of state-of-the-art two semantic segmentation models, and two object detectors using public databases. We also examined the effect of the breast density on the accuracy of localizing and segmenting the breast masses. Our proposed model shows superior performance compared to the other DL methods in detecting and segmenting masses, especially for heterogeneously dense and dense MG images.