Recalibration in Validation Studies of Diabetes Risk Prediction Models: A Systematic Review

Abstract

Background: Poor performance of risk prediction models in a new setting is common. Recalibration methods aim to improve the prediction performance of a model in a validation population, however the extent of its application in the validation of diabetes risk prediction models is not yet known. Methods: We critically reviewed published validation studies of diabetes prediction models, selected from five recent comprehensive systematic reviews and database searches. Common recalibration techniques applied were described and the extent to which recalibration and impacts were reported analysed. Results: Of the 236 validations identified, 22.9% (n = 54) undertook recalibration on existent models in the validation population. The publication of these studies was consistent from 2008. Only incident diabetes risk prediction models were validated, and the most commonly validated Framingham offspring simple clinical risk model was the most recalibrated of the models, in 4 studies (7.4%). Conclusions: This review highlights the lack of attempt by validation studies to improve the performance of the existent models in new settings. Model validation is a fruitless exercise if the model is not recalibrated or updated to allow for greater accuracy. This halts the possible implementation of an existent model into routine clinical care. The use of recalibration procedures should be encouraged in all validation studies, to correct for the anticipated drop in model performance.