Hybrid vision transformer and graph neural network model with region-adaptive attention for enhanced skin cancer prediction

Abstract

A well-known and potentially lethal skin cancer requires prompt detection and diagnosis. Complex spatial linkages and global contextual information in skin lesion photos challenge CNNs and other deep learning methods. Given these restrictions, we present a Hybrid Vision Transformer (ViT) with a Graph Neural Network (GNN) and Region-Adaptive Attention to diagnose skin cancer. The ViT branch captures dermoscopy image global dependencies, whereas the GNN enhances features by exploiting lesions’ spatial relationships. Region-Adaptive Attention improves lesion categorization by dynamically improving feature extraction in diagnostically relevant locations. Our paradigm for multi-scale lesion analysis accounts for lesion size, color, and texture changes. Meta-learning methods refine the proposed model to make it generalizable across skin tones and imaging settings. Our model outperformed state-of-the-art deep learning algorithms on benchmark skin cancer datasets. The architecture improves classification accuracy and interpretability, making it a promising clinical dermatology tool.