A Scalable Big Data-Driven Distributed Deep Learning Framework for Breast Cancer Diagnosis Using Big Data Analytics

Abstract

The accurate and early detection of breast cancer remains a significant challenge in medical diagnostics, primarily due to the complexity of histopathological images and the large volume of data involved. This paper presents a novel hybrid deep learning framework that leverages Big Data Analytics (BDA) and Convolutional Neural Networks (CNNs) to enhance the accuracy of breast cancer detection. The proposed system integrates three robust deep learning architectures (VGG16, VGG19, and ResNet50) trained in parallel across distributed nodes using Apache Spark, thereby accelerating computation and enabling scalable learning. This study used the BreakHis dataset, which contains 15,918 original images collected at four magnifications. To enhance generalization and class balance, extensive data augmentation and patch extraction were applied, which expanded the dataset to approximately 275,000 training samples. The hybrid model demonstrated high performance in classification tasks, achieving high precision, recall, and F1-scores compared to existing benchmarks. Key performance indicators, such as accuracy, specificity, and sensitivity, confirm the effectiveness of the model in distinguishing between benign and malignant cases. Unlike traditional monolithic CNN approaches, the proposed system leverages distributed processing to reduce training time while efficiently handling massive datasets.