Change point analysis for detecting vaccine safety signals

Abstract

It is important to detect signals of abrupt changes in adverse event reporting in order to notice public safety concerns and take prompt action, especially for vaccines under national immunization programs. In this study, we assessed the applicability of change point analysis (CPA) for signal detection in vaccine safety surveillance. The performances of three CPA methods, namely Bayesian change point analysis, Taylor’s change point analysis (Taylor-CPA), and environmental time series change point detection (EnvCpt), were assessed via simulated data with assumptions for the baseline number of events and degrees of change. The analysis was validated using the Korea Adverse Event Reporting System (KAERS) database. In the simulation study, the Taylor-CPA method exhibited better results for the detection of a change point (accuracy of 96% to 100%, sensitivity of 7% to 100%, specificity of 98% to 100%, positive predictive value of 25% to 85%, negative predictive value of 96% to 100%, and balanced accuracy of 53% to 100%) than the other two CPA methods. When the CPA methods were applied to reports of syncope or dizziness following human papillomavirus (HPV) immunization in the KAERS database, Taylor-CPA and EnvCpt detected a change point (Q2/2013), which was consistent with actual public safety concerns. CPA can be applied as an efficient tool for the early detection of vaccine safety signals.