Activation Heatmap-Guided FT-MultiCNN: Advancing Skin Cancer Classification Through Transfer Learning

Abstract

Cancer of the skin is one of the most prevalent and malignant diseases known to mankind. The most effective treatment for skin cancer is early and accurate detection. Skin lesions which are often complex in structure and diverse in appearance make it not easy for traditional machine learning methods to precisely obtain features and recognize them. We propose a Multi-modal CNN for Skin Cancer detection and classification to address these difficulties. Our approach used a Fine-tune Custom CNN multi-model (FT-MultiCNN) to handle and fuse image data and metadata. The processed images were then employed for input to the FT-MultiCNN model beside patient’s metadata as well as the extracted Class Activation Heat-map (CAM) features. To identify skin lesions, a CNN was used using parallel processing architecture and multimodal fusion. This model was trained and tested on a public ISIC dataset, and its performance was assessed using cross-validation and compared to other leading approaches. Our model beat existing machine learning and transfer learning models in accuracy and recall, with 0.96 ± 0.25 accuracy and 0.94 ±.34 recall, indicating robust performance. The experiments show exceptional classification capabilities, producing cutting-edge outcomes in identifying diverse skin cancer forms. This work provides a promising line for automated non-invasive screening of skin cancer, and paves the way for the promise of multimodal deep learning in dermatology.