The impact on sample sizes of studies if the significance level is changed from an α of 0.05 to 0.005

Abstract

Background: The statistical significance α = 0.05 is the cut-off point used to decide whether a hypothesis is statistically significant. When p-value is less than 0.05, we reject the null hypothesis. Although this criterion has been used for almost a century to generate new knowledge, there is currently an international discussion about the need to decrease the significance to α = 0.005. Aim: To determine the effects that changing the p value would have on the sample size of different types of studies. Material and Methods: A series of formulas for calculating the sample size of cross-sectional and comparative studies were used to create case scenarios. Results: By changing α = 0.05 to α = 0.005, the sample sizes in cross-sectional studies would double and in comparative studies would increase between 60% and 70%, depending on the statistical power chosen. Conclusions: Considering the sample size implications, the change in the level of significance would have important effects on the Chilean science. The cost of a randomized clinical trial could increase by at least 27% to 32%. This increase could be similar for cross-sectional studies. With an investment of less than 0.4% of gross domestic product in science and technology, national scientific research would become more expensive, distributing the few available resources among fewer projects. This effect should be considered in any discussion about national budget for science and technology.