Assessing evidence for replication: A likelihood-based approach

Abstract

How to evaluate replications is a fundamental issue in experimental methodology. We develop a likelihood-based approach to assessing evidence for replication. In this approach, the design of the original study is used to derive an estimate of a theoretically interesting effect size. A likelihood ratio is then calculated to contrast the match of two models to the data from the replication attempt: (1) a model based on the derived theoretically interesting effect size, and (2) a null model. This approach provides new insights not available with existing methods of assessing replication. When applied to data from the Replication Project (Open Science Collaboration, 2015), the procedure indicates that a large portion of the replications failed to find evidence for a theoretically interesting effect.