Everything hacked? What is the evidential value of the experimental public administration literature?

Abstract

The rise of behavioral public administration provides new perspectives – especially from a psychological point of view – to understand public administration theories and the growing interest in using experiments to enhance the internal validity of empirical studies. However, psychology and other social sciences are undergoing a replication crisis where experimental results often do not replicate. One reason for the limited replicability is the publication bias sparked by journals’ preference for significant effects and the resulting incentive to create significant results. This study employs a meta-analytical approach to examine the evidential value of experimental evidence in public administration. It uses the p-curve method to test whether this body of research is dominated by selectively reporting significant results. The analysis includes 172 statistically significant findings published in top public administration journals and shows that the distribution of p values of these findings is right-skewed. Such a distribution indicates that the experimental public administration research contains evidential value, which means it is not solely the result of selective reporting of significant results. Although the analysis shows a good sign, we discuss important practices to further strengthen the validity and reliability of experimental methods in public administration.