Multimodal Diagnostic Approach for Osteosarcoma and Bone Callus Using Hyperspectral Imaging and Deep Learning

Abstract

Distinguishing osteosarcoma from bone callus remains a clinical challenge due to their morphological similarities. This study proposes J-CAN, a multimodal deep learning framework integrating hyperspectral imaging (HSI) and H&E-stained pathology for rapid and accurate classification. The HSI system captures 176 spectral bands (400–1000 nm), providing molecular-level insights. MobileNetV2 extracts spatial features, while 1D-CNN processes spectral signatures. A self-attention mechanism enhances feature selection, prioritizing key spectral and spatial characteristics to improve classification performance. Experimental results show that J-CAN outperforms conventional models, including LSTM, SVM, and 1D-CNN, achieving 87.33% accuracy, 89.07% sensitivity, and 85.49% specificity. These findings demonstrate the potential of HSI-driven deep learning for clinical pathology, enabling efficient, automated osteosarcoma diagnosis. This approach enhances diagnostic precision and provides a valuable tool for pathologists, addressing the limitations of traditional histopathological assessments and improving the differentiation between osteosarcoma and bone callus.