Let’s Put Our Money Where Our Mouth Is

Abstract

A number of scholars recently have argued for fundamental changes in the way psychological scientists conduct and report research. The behavior of researchers is influenced partially by incentive structures built into the manuscript evaluation system, and change in researcher practices will necessitate a change in the way journal reviewers evaluate manuscripts. This article outlines specific recommendations for reviewers that are designed to facilitate open data reporting and to encourage researchers to disseminate the most generative and replicable studies. These recommendations include changing the way reviewers respond to imperfections in empirical data, focusing less on individual tests of statistical significance and more on meta-analyses, being more open to null findings and failures to replicate previous research, and attending carefully to the theoretical contribution of a manuscript in addition to its methodological rigor. The article also calls for greater training and guidance for reviewers so that they can evaluate research in a manner that encourages open reporting and ultimately strengthens our science.