Artificial Intelligence for Improved Patient Outcomes—The Pragmatic Randomized Controlled Trial Is the Secret Sauce

Abstract

Artificial Intelligence (AI) has exploded in the media for both its astonishing power and disturbing weaknesses. Of the potential applications of AI that are most likely to benefit society, most thought leaders point to medicine. Yet, to date, we have almost no rigorous evidence that AI improves patient health outcomes [1-3]. Why is there a dearth of evidence? What needs to change? First, let’s look at what has not been working. Most applications of AI in healthcare have had no outcome evaluation or had one with an inadequate study design that would not result in reproducible research [4]. Many AI evaluations are based on observational studies that are so profoundly biased that they provide no compelling evidence that the AI tool is safe or improves patient outcomes [4]. This is compounded by the problem that some junior AI researchers are new to clinical research and are unaware that their outcome evaluations are fatally flawed.