Development and external validation of an admission risk prediction model after treatment from early intervention in psychosis services

Abstract

Early Intervention in psychosis (EIP) teams are the gold standard treatment for first-episode psychosis (FEP). EIP is time-limited and clinicians are required to make difficult aftercare decisions that require weighing up individuals’ wishes for treatment, risk of relapse, and health service capacity. Reliable decision-making tools could assist with appropriate resource allocation and better care. We aimed to develop and externally validate a readmission risk tool for application at the point of EIP discharge. All persons from EIP caseloads in two NHS Trusts were eligible for the study. We excluded those who moved out of the area or were only seen for assessment. We developed a model to predict the risk of hospital admission within a year of ending EIP treatment in one Trust and externally validated it in another. There were n = 831 participants in the development dataset and n = 1393 in the external validation dataset, with 79 (9.5%) and 162 (11.6%) admissions to inpatient hospital, respectively. Discrimination was AUC = 0.76 (95% CI 0.75; 0.77) in the development dataset and AUC = 0.70 (95% CI 0.66; 0.75) in the external dataset. Calibration plots in external validation suggested an underestimation of risk in the lower predicted probabilities and slight overestimation at predicted probabilities in the 0.1–0.2 range (calibration slope = 0.86, 95% CI 0.68; 1.05). Recalibration improved performance at lower predicted probabilities but underestimated risk at the highest range of predicted probabilities (calibration slope = 1.00, 95% CI 0.79; 1.21). We showed that a tool for predicting admission risk using routine data has good performance and could assist clinical decision-making. Refinement of the model, testing its implementation and further external validation are needed.