A tool for assessing the feasibility of comparative effectiveness research

Abstract

Background: Comparative effectiveness research (CER) provides actionable information for health care decision-making. Randomized clinical trials cannot provide the patients, time horizons, or practice settings needed for all required CER. The need for comparative assessments and the infeasibility of conducting randomized clinical trials in all relevant areas is leading researchers and policy makers to non-randomized, retrospective CER. Such studies are possible when rich data exist on large populations receiving alternative therapies that are used as-if interchangeably in clinical practice. This setting we call “empirical equipoise.” Objectives: This study sought to provide a method for the systematic identification of settings it in which it is empirical equipoise that offers promised non-randomized CER. Methods: We used a standardizing transformation of the propensity score called “preference” to assess pairs of common treatments for uncomplicated community-acquired pneumonia and new-onset heart failure in a population of low-income elderly people in Pennsylvania, for whom we had access to de-identified insurance records. Treatment pairs were considered suitable for CER if at least half of the dispensings of each treatment-pair member fell within a preference range of 30% to 70%. Results: Among 3889 community-acquired pneumonia patients, insurance claims histories were sufficiently similar in seven drug pairs to suggest that observational CER might be effective. Relapse appears to have been less common in levofloxacin recipients than in similar patients given other products. In 6035 heart failure patients, metoprolol, carvedilol, and atenolol were employed in patients with similar claims histories, and thus might be suitable for observational CER. The long-acting succinate formulation of metoprolol had lower failure rates in head-to-head comparisons with all other beta-blockers. Both findings are candidates for further investigation. Confounding by unmeasured factors operating in the same manner as the measured covariates would not have produced the apparent superiority of levofloxacin, which was given to people in poorer respiratory health. The baseline covariate distributions of persons starting beta-blockers suggest only that carvedilol recipients were healthier than others. Conclusion: A straightforward algorithm can identify empirical equipoise, in which prescribers as a group seem evenly divided on the merits of alternative therapies. This is the setting in which CER may be most necessary and is likely to be most accurate. The imbalances identified by propensity models can identify situations in which the results of screening analyses may be biased in the direction of the observed effect.