Biases Affecting Human Decision Making in AI-Supported Second Opinion Settings

Abstract

In this paper we focus on a still neglected consequence of the adoption of AI in diagnostic settings: the increase of cases in which a human decision maker is called to settle a divergence between a human doctor and the AI, i.e., second opinion requests. We designed a user study, involving more than 70 medical doctors, to understand if the second opinions are affected by the first ones and whether the decision makers tend to trust the human interpretation more than the machine’s one. We observed significant effects on decision accuracy and a sort of “prejudice against the machine”, which varies with respect to the respondent profile. Some implications for sounder second opinion settings are given in the light of the results of this study.