Accounting for Multiplicity in Inference on Economics Journal Rankings

Abstract

Nearly all journal ranking analyses assume that rank statistics of journal quality are deterministic, yet they are clearly random. The only study to recognize ranking uncertainty is Stern (2013), which calculates standard errors for a ranking of five-year impact factors for 232 economics journals and performs inference using a series of univariate t-tests. We revisit the Stern study but perform multivariate inference to control the overall error rate of the testing procedure. The results are compared and differences highlighted.