Comparison of mRNA vaccine effectiveness against COVID-19-associated hospitalization by vaccination source: Immunization information systems, electronic medical records, and self-report—IVY Network, February 1–August 31, 2022

Abstract

Background: Accurate determination of COVID-19 vaccination status is necessary to produce reliable COVID-19 vaccine effectiveness (VE) estimates. Data comparing differences in COVID-19 VE by vaccination sources (i.e., immunization information systems [IIS], electronic medical records [EMR], and self-report) are limited. We compared the number of mRNA COVID-19 vaccine doses identified by each of these sources to assess agreement as well as differences in VE estimates using vaccination data from each individual source and vaccination data adjudicated from all sources combined. Methods: Adults aged ≥18 years who were hospitalized with COVID-like illness at 21 hospitals in 18 U.S. states participating in the IVY Network during February 1–August 31, 2022, were enrolled. Numbers of COVID-19 vaccine doses identified by IIS, EMR, and self-report were compared in kappa agreement analyses. Effectiveness of mRNA COVID-19 vaccines against COVID-19-associated hospitalization was estimated using multivariable logistic regression models to compare the odds of COVID-19 vaccination between SARS-CoV-2-positive case-patients and SARS-CoV-2-negative control-patients. VE was estimated using each source of vaccination data separately and all sources combined. Results: A total of 4499 patients were included. Patients with ≥1 mRNA COVID-19 vaccine dose were identified most frequently by self-report (n = 3570, 79 %), followed by IIS (n = 3272, 73 %) and EMR (n = 3057, 68 %). Agreement was highest between IIS and self-report for 4 doses with a kappa of 0.77 (95 % CI = 0.73–0.81). VE point estimates of 3 doses against COVID-19 hospitalization were substantially lower when using vaccination data from EMR only (VE = 31 %, 95 % CI = 16 %–43 %) than when using all sources combined (VE = 53 %, 95 % CI = 41 %–62%). Conclusion: Vaccination data from EMR only may substantially underestimate COVID-19 VE.