Comparison of the Test-negative Design and Cohort Design With Explicit Target Trial Emulation for Evaluating COVID-19 Vaccine Effectiveness

Abstract

Background: Observational studies are used for estimating vaccine effectiveness under real-world conditions. The practical performance of two common approaches—cohort and test-negative designs—need to be compared for COVID-19 vaccines. Methods: We compared the cohort and test-negative designs to estimate the effectiveness of the BNT162b2 vaccine against COVID-19 outcomes using nationwide data from the United States Department of Veterans Affairs. Specifically, we (1) explicitly emulated a target trial using follow-up data and evaluated the potential for confounding using negative controls and benchmarking to a randomized trial, (2) performed case–control sampling of the cohort to confirm empirically that the same estimate is obtained, (3) further restricted the sampling to person–days with a test, and (4) implemented additional features of a test-negative design. We also compared their performance in limited datasets. Results: Estimated BNT162b2 vaccine effectiveness was similar under all four designs. Empirical results suggested limited residual confounding by healthcare-seeking behavior. Analyses in limited datasets showed evidence of residual confounding, with estimates biased downward in the cohort design and upward in the test-negative design. Conclusion: Vaccine effectiveness estimates under a cohort design with explicit target trial emulation and a test-negative design were similar when using rich information from the VA healthcare system, but diverged in opposite directions when using a limited dataset. In settings like ours with sufficient information on confounders and other key variables, the cohort design with explicit target trial emulation may be preferable as a principled approach that allows estimation of absolute risks and facilitates interpretation of effect estimates.