Programmed Inefficiencies in DSS-Supported Human Decision Making

Abstract

Machine learning–based decision support systems (DSS) are attracting the interest of the medical community. Their usage, however, could have deep consequences in terms of biasing the doctor’s interpretation of a case through automation bias and deskilling. In this work we address the design of DSS with the goal of minimizing these biases through the design and implementation of programmed inefficiencies (PIs), that is, features with the stated purpose of making the reliance of the human doctor on the DSS less obvious (or more difficult). We illustrate this concept by presenting a real-life medical DSS, called DataWise, embedding different PIs and currently undergoing iterative prototyping and testing with the medical users in two clinical settings. We describe the main features of DataWise, and show how different PIs have been conveyed by prompting doctors for multiple input and using qualitative visualizations instead of precise, but possibly misleading, indications. Finally, we discuss the implications of this design approach to naturalistic decision making, especially in life-saving domains like medicine is.