Regional validation and recalibration of clinical predictive models for patients with acute heart failure

Abstract

Background--Heart failure clinical practice guidelines recommend applying validated clinical predictive models (CPMs) to support decision making. While CPMs are now widely available, the generalizability of heart failure CPMs is largely unknown. Methods and Results--We identified CPMs derived in North America that predict mortality for patients with acute heart failure and validated these models in different world regions to assess performance in a contemporary international clinical trial (N=4133) of patients with acute heart failure treated with guideline-directed medical therapy. We performed independent external validations of 3 CPMs predicting in-hospital mortality, 60-day mortality, and 1-year mortality, respectively. CPM discrimination decreased in all regional validation cohorts. The median change in area under the receiver operating curve was -0.09 (range -0.05 to -0.23). Regional calibration was highly variable (90th percentile of absolute difference between smoothed observed and predicted values range < 1% to > 50%). Calibration remained poor after global recalibrations; however, region-specific recalibration procedures significantly improved regional performance (recalibrated 90th percentile of absolute difference range < 1% to 5% across all regions and all models). Conclusions--Acute heart failure CPM discrimination and calibration vary substantially across different world regions; regionspecific (as opposed to global) recalibration techniques are needed to improve CPM calibration.