Attach importance of the bootstrap t test against Student’s t test in clinical epidemiology: a demonstrative comparison using COVID-19 as an example

Abstract

Student’s t test is valid for statistical inference under the normality assumption or asymptotically. By contrast, although the bootstrap t test was proposed in 1993, it is seldom adopted in medical research. We aim to demonstrate that the bootstrap t test outperforms Student’s t test under normality in data. Using random data samples from normal distributions, we evaluated the testing performance, in terms of true-positive rate (TPR) and false-positive rate and diagnostic abilities, in terms of the area under the curve (AUC), of the bootstrap t test and Student’s t test. We explore the AUC of both tests with varying sample size and coefficient of variation. We compare the testing outcomes using the COVID-19 serial interval (SI) data in Shenzhen and Hong Kong, China, for demonstration. With fixed TPR, the bootstrap t test maintained the equivalent accuracy in TPR, but significantly improved the true-negative rate from the Student’s t test. With varying TPR, the diagnostic ability of bootstrap t test outperformed or equivalently performed as Student’s t test in terms of the AUC. The equivalent performances are possible but rarely occur in practice. We find that the bootstrap t test outperforms by successfully detecting the difference in COVID-19 SI, which is defined as the time interval between consecutive transmission generations, due to sex and non-pharmaceutical interventions against the Student’s t test. We demonstrated that the bootstrap t test outperforms Student’s t test, and it is recommended to replace Student’s t test in medical data analysis regardless of sample size.