The random effects prep continues to mispredict the probability of replication

Abstract

In their reply, Lecoutre and Killeen (2010) argue for a random effects version of prep, in which the observed effect from one experiment is used to predict the probability that an effect from a different but related experiment will have the same sign. They present a figure giving the impression that this version of prep accurately predicts the probability of replication. We show that their results are incorrect and conceptually limited, even when corrected. We then present a meaningful evaluation of the random effects prep as a predictor and find that, as with the fixed effects prep, it performs very poorly. © 2010 The Psychonomic Society, Inc.