Explainable AI for Clinical Decision Support: SHAP and Grad-CAM in Text- and Image-Based Medical Predictions

Abstract

The integration of Artifcial Intelligence (AI), par- ticularly deep learning, into clinical workfows promises to revo- lutionize medical diagnostics. High-performance models can now analyze medical images and clinical text with accuracy rivaling, and sometimes exceeding, human experts. However, the “black- box” nature of these complex models is a signifcant barrier to their widespread clinical adoption. Physicians require transpar- ent and interpretable reasoning to trust AI-driven predictions for high-stakes decisions. This survey addresses this critical need by providing a focused review of Explainable AI (XAI) techniques tailored for medical applications. We move beyond general discussions of interpretability to conduct a detailed analysis of two of the most prominent and practical post-hoc XAI methods: SHapley Additive exPlanations (SHAP) for text-based predictions from electronic health records, and Gradient-weighted Class Activation Mapping (Grad-CAM) for image-based predictions from modalities like CT and MRI. We introduce a taxonomy of XAI and situate these methods within it, reviewing their mechanisms, applications, and limitations. By focusing on this multimodal approach, this paper serves as a practical guide for researchers and clinicians aiming to develop, evaluate, and deploy the next generation of trustworthy, transparent, and effective AI- powered clinical decision support systems.