Publication Bias in Meta-Analysis: Confidence Intervals for Rosenthal’s Fail-Safe Number

Abstract

The purpose of the present paper is to assess the efficacy of confidence intervals for Rosenthal’s fail-safe number. Although Rosenthal’s estimator is highly used by researchers, its statistical properties are largely unexplored. First of all, we developed statistical theory which allowed us to produce confidence intervals for Rosenthal’s fail-safe number. This was produced by discerning whether the number of studies analysed in a meta-analysis is fixed or random. Each case produces different variance estimators. For a given number of studies and a given distribution, we provided five variance estimators. Confidence intervals are examined with a normal approximation and a nonparametric bootstrap. The accuracy of the different confidence interval estimates was then tested by methods of simulation under different distributional assumptions. The half normal distribution variance estimator has the best probability coverage. Finally, we provide a table of lower confidence intervals for Rosenthal’s estimator.