Is economics self-correcting? Replications in the American Economic Review

Abstract

This paper reviews the impact of replications published as comments in the American Economic Review between 2010 and 2020. We examine their citations and influence on the original papers' (OPs) subsequent citations. Our results show that comments are barely cited, and they do not affect the OP's citations—even if the comment diagnoses substantive problems. Furthermore, we conduct an opinion survey among replicators and authors and find that there often is no consensus on whether the OP's contribution sustains. We conclude that the economics literature does not self-correct, and that robustness and replicability are hard to define in economics.