Enhancing AI Clinical Decision Support Trust: Design Workshop Insights from General Practitioners

Abstract

Artificial Intelligence (AI) predictive models are increasingly integrated into Clinical Decision Support Systems (CDSS). However, real-world implementation is lagging due to a lack of trust and acceptance by healthcare providers. To investigate factors related to trust and acceptance of AI-based CDSS, we conducted a workshop with General Practitioners to explore user, model, and organizational factors that could affect trust and recommendation acceptance. The workshop discussions revealed that while explainability is crucial for understanding AI decisions, current explainable AI (XAI) visualizations appeared to provide limited value to clinicians without technical backgrounds. Participants suggested that transparency in model training information, including data sources and elements, combined with clear alignment between model decisions and established clinical research findings and guidelines, would enhance their trust and acceptance of AI-based recommendations. These insights provide valuable direction for developing user interfaces that enhance clinician trust and acceptance of AI-based CDSS in clinical practice.