Naming and Diffusing the Understanding Objection in Healthcare Artificial Intelligence

Abstract

Informed consent is often argued to be one of the more significant potential problems for the implementation and widespread onboarding of artificial intelligence (AI) and machine learning in healthcare decision-making. This is because of the concern revolving around whether, and to what degree, patients can understand what contributes to the decision-making process when an algorithm is involved. In this paper, I address what I call the Understanding Objection, which is the idea that AI systems will cause problems for the informational criteria involved in proper informed consent. I demonstrate that collaboration with clinicians in a human-in-the-loop partnership can alleviate these concerns around understanding, regardless how one conceptualizes the scope of understanding. Importantly, I argue that the human clinicians must be the second reader in the partnership to avoid institutional deference to the machine and best promote clinicians as the experts in the process.