The importance of falsification endpoints in observational studies of vaccination to prevent severe disease: A critique of a harm-benefit analysis of BNT162b2 vaccination of 5- to 11-year-olds

Abstract

We explore one systematic review and meta-analysis of both observational and randomized studies examining COVID-19 vaccines in 5- to 11-year-olds, which reported substantial benefits associated with vaccinating this age group. We discuss the limitations of the individual studies that were used to estimate vaccination benefits. The review included five observational studies that evaluated vaccine effectiveness (VE) against COVID-19 severe disease or hospitalization. All five studies failed to adequately assess differences in underlying health between vaccination groups. In terms of vaccination harms, looking only at the randomized studies, a significantly higher odds of adverse events was identified among the vaccinated compared with the unvaccinated. Observational studies are at risk of overestimating the effectiveness of vaccines against severe disease if healthy vaccinee bias is present. Falsification endpoints can provide valuable information about underlying healthy vaccinee bias. Studies that have not adequately ruled out bias due to better health among the vaccinated or more vaccinated should be viewed as unreliable for estimating the VE of COVID-19 vaccination against severe disease and mortality. Existing systematic reviews that include observational studies of the COVID-19 vaccine in children may have overstated or falsely inferred vaccine benefits due to unidentified or undisclosed healthy vaccinee bias.