Breast Cancer Classification Through Transfer Learning with Vision Transformer, PCA, and Machine Learning Models

Abstract

Breast cancer is a leading cause of death among women worldwide, making early detection crucial for saving lives and preventing the spread of the disease. Deep Learning and Machine Learning techniques, coupled with the availability of diverse breast cancer datasets, have proven to be effective in assisting healthcare practitioners worldwide. Recent advancements in image classification models, such as Vision Transformers and pretrained models, offer promising avenues for breast cancer imaging classification research. In this study, we employ a pretrained Vision Transformer (ViT) model, specifically trained on the ImageNet dataset, as a feature extractor. We combine this with Principal Component Analysis (PCA) for dimensionality reduction and evaluate two classifiers, namely a Multilayer Perceptron (MLP) and a Support Vector Machine (SVM), for breast mammogram image classification. The results demonstrate that the transfer learning approach using ViT, PCA, and an MLP classifier achieves an average accuracy, precision, recall, and F1-score of 98% for the DSMM dataset and 95% for the INbreast dataset, considering the same metrics which are comparable to the current state-of-the-art.