How Displaying AI Confidence Affects Reliance and Hybrid Human-AI Performance

Abstract

Confidence signals are often used in human interactions to communicate the likelihood of a decision being correct. Similarly, confidence may also be used to indicate the reliability of advice given by an AI. While previous work on explainable AI (XAI) has explored the effect of AI confidence on AI-advice adoption and joint accuracy of the human-AI team, most studies use AI-assistants that exceed human performance. It is unclear how displaying the confidence interacts with the accuracy of the AI. We conduct a comprehensive investigation of the effect of displaying AI confidence on two factors: 1) the accuracy of AI-assisted decision making, and 2) reliance on the AI's assistance. We conduct two behavioral experiments, one where participants were shown AI confidence, and another where no confidence ratings were shown. Our work goes beyond the typical focus on high accuracy AI assistants. In both experiments, participants were assisted by one of three AI classifiers of varying accuracy. Our results demonstrate that displaying AI confidence increases joint accuracy when people are assisted by a classifier that is better than humans on average. Conversely, when assisted by a classifier with performance worse than an average human, joint accuracy was better when no AI confidence was displayed. However, for the adoption of AI advice we observed the opposite pattern: people rely more on a higher accuracy classifier that does not display confidence compared to one that does, and people rely more on a lower accuracy classifier that does display AI confidence compared to one that does not.