Sensitivity Analysis for Unmeasured Confounding in Medical Product Development and Evaluation Using Real World Evidence

Abstract

The American Statistical Association Biopharmaceutical Section (ASA BIOP) scientific working group on real-world evidence (RWE) has been making continuous, extended efforts toward a goal of supporting and advancing regulatory science with respect to clinical studies intended to use real-world data for evidence generation for the purpose of medical product development and evaluation (i.e., RWE studies). In 2023, the working group published a manuscript delineating challenges and opportunities in constructing estimands for RWE studies following the framework in ICH E9(R1) guidance on estimand and sensitivity analysis. As a follow-up task, we describe the other issue—sensitivity analysis. Although the FDA’s definition of RWE studies includes randomized trials using RWD as a primary source of evidence generation such as pragmatic trials, here we focus on non-randomized RWE studies and the issue of unmeasured confounding which is a major source of bias for most RWE studies. We review the availability and applicability of sensitivity analysis methods for different types of unmeasured confounding. We also provide some practical and regulatory considerations on using sensitivity analysis for such RWE studies. Then, we use a plasmode case study to demonstrate realistic ways to interpret sensitivity analysis results in supporting regulatory decision-making. We conclude with a brief discussion.