Overview of basic design recommendations for user-centered explanation interfaces for AI-based clinical decision support systems: A scoping review

Abstract

Objective: The application of artificial intelligence (AI)-based clinical decision support systems (CDSS) in the healthcare domain is still limited. End-users’ difficulty understanding how the outputs of opaque black AI models are generated contributes to this. It is still unknown which explanations are best presented to end users and how to design the interfaces they are presented in (explanation user interface, XUI). This article aims to provide an overview of recommendations for the user-centered design of XUIs from the scientific literature. Methods: A scoping review was conducted to identify recommendations for the design of XUIs. Articles published between 2017 and 2022 in English or German, presenting original research or literature reviews, focusing on XUIs for end users or domain experts, which are intended for presentation in graphical user interfaces and from which recommendations could be extracted were included in the review. Articles were retrieved from Scopus, Web of Science, IEEE Explore, PubMed, ACM Digital Library, and PsychInfo. A mind map was created to organize and summarize the identified recommendations. Results: From the 47 included articles, 240 recommendations for the user-centered design were extracted. The organization in a mind map resulted in 64 summarized recommendations. Conclusion: This review provides a synopsis of basic recommendations for the user-centered design of XUIs, focusing on the healthcare domain. During the analysis of the articles, it became clear that no specific and directly implementable design recommendations for AI-based CDSS can be given, but only basic recommendations for raising awareness about the user-centered design of XUIs.