Development and External Validation of the International Early Warning Score for Improved Age- and Sex-Adjusted In-Hospital Mortality Prediction in the Emergency Department

Abstract

Objectives: Early Warning Scores (EWSs) have a great potential to assist clinical decision-making in the emergency department (ED). However, many EWS contain methodological weaknesses in development and validation and have poor predictive performance in older patients. The aim of this study was to develop and externally validate an International Early Warning Score (IEWS) based on a recalibrated National Early warning Score (NEWS) model including age and sex and evaluate its performance independently at arrival to the ED in three age categories (18-65, 66-80, > 80 yr). Design: International multicenter cohort study. Setting: Data was used from three Dutch EDs. External validation was performed in two EDs in Denmark. Patients: All consecutive ED patients greater than or equal to 18 years in the Netherlands Emergency department Evaluation Database (NEED) with at least two registered vital signs were included, resulting in 95,553 patients. For external validation, 14,809 patients were included from a Danish Multicenter Cohort (DMC). Measurements and Main Results: Model performance to predict in-hospital mortality was evaluated by discrimination, calibration curves and summary statistics, reclassification, and clinical usefulness by decision curve analysis. In-hospital mortality rate was 2.4% (n = 2,314) in the NEED and 2.5% (n = 365) in the DMC. Overall, the IEWS performed significantly better than NEWS with an area under the receiving operating characteristic of 0.89 (95% CIs, 0.89-0.90) versus 0.82 (0.82-0.83) in the NEED and 0.87 (0.85-0.88) versus 0.82 (0.80-0.84) at external validation. Calibration for NEWS predictions underestimated risk in older patients and overestimated risk in the youngest, while calibration improved for IEWS with a substantial reclassification of patients from low to high risk and a standardized net benefit of 5-15% in the relevant risk range for all age categories. Conclusions: The IEWS substantially improves in-hospital mortality prediction for all ED patients greater than or equal to18 years.