Development and Validation of a Clinical Prediction Tool for Seasonal Influenza Vaccination in England

Abstract

Importance: Timely identification of patients likely to miss seasonal influenza vaccination (SIV) could help health care practitioners tailor services and gain efficiency. Objective: To develop and validate a predictive model of SIV uptake among at-risk adults. Design, Setting, and Participants: This prognostic study constructed a prediction model for vaccine uptake by adults at increased risk of influenza-associated complications. Drawing from the Clinical Practice Research Datalink database's records of primary care data of 324284 adults routinely collected at general practices across England from January 2011 to December 2016, logistic regression models were trained on data from patients registered from January 2012 to December 2013 and validated with out-of-sample data from patients registered from January 2015 to December 2016. Data were extracted from the database December 2018 and analyzed between September 2019 and December 2019. Exposures: Covariates included sex, age, race/ethnicity, smoking status, socioeconomic status, previous pneumococcal vaccination, prior season SIV uptake, and clinical risk conditions. Main Outcomes and Measures: The main outcome was patient-level SIV uptake. Model performance was measured via misclassification rate, Brier score, sensitivity, specificity, and area under the curve. Results: The training data sets consisted of 324284 (aged 18 to 64 years) and 186426 (aged 65 years or older) patients. The mean (SD) age in the training data among patients aged 18 to 64 years was 45 (13) years; 161487 (49.8%) were women, and 102133 (31.5%) were categorized as white. Among patients aged 65 years or older, the mean (SD) age was 77 (8) years; 96169 (51.6%) were women, and 64996 (34.9%) were categorized as white. The validation data sets consisted of 35210 patients aged 18 to 64 years and 25497 aged 65 years or older. The mean (SD) age in the validation data set among patients aged 18 to 64 years was 42 (14) years; 17 296 (49.1%) were women, and 13 346 (37.9%) were categorized as white. Among patients aged 65 years or older, the mean (SD) age was 73 (8) years; 13 135 (51.5%) were women, and 9641 (37.8) were categorized as white. Among patients aged 18 to 64 years, SIV uptake was 35.9% (95% CI, 35.7%-36.0%) and 32.6% (95% CI, 32.1%-33.1%) for the training and validation data sets, respectively. Among patients aged 65 years or older, SIV uptake was 83.1% (95% CI, 82.9%-83.2%) and 76.1% (95% CI, 75.5%-76.6%) for the training and validation data sets, respectively. Prior season SIV uptake and pneumococcal vaccination status were the best predictors of SIV uptake. Predicted SIV uptake probabilities for patients aged 18 to 64 years were reliable, but biased toward underpredicting, whereas, among patients aged 65 years or older, they were variable and biased toward overpredicting. Briefly, in out-of-sample validation among patients aged 18 to 64 years, misclassification rates were 0.163 to 0.164, Brier scores were 0.124 to 0.125, area under the receiver operating characteristic curve values ranged from 0.876 to 0.877, sensitivity ranged from 0.705 to 0.720, and specificity ranged from 0.896 to 0.902. In patients aged 65 years or older, misclassification rates were 0.120 to 0.125, Brier scores were 0.0953 to 0.0959, area under the receiver operating characteristic curve was 0.877, sensitivity ranged from 0.919 to 0.936, and specificity ranged from 0.680 to 0.753. Conclusions and Relevance: This study suggests that data obtained from primary care records could accurately predict SIV uptake among at-risk adults. Further research is needed to assess the feasibility and efficacy of implementing this model in clinical settings..