P-values don’t measure evidence

Abstract

Despite much writing about the uses and misuses of p-values, one point is frequently misunderstood: p-values do not measure evidence. Even so, the statistics literature often says that p-values do measure evidence. The purpose of the present article is to argue through several examples that p-values do not measure evidence. This article is not about other aspects of p-values. Specifically, it is not about sharp cutoffs, effect sizes, decision making, or other potential measures of evidence. We felt compelled to write about whether p-values measure evidence because it is our opinion that misunderstanding evidence and how to measure it contributes to misunderstanding science and to the so-called replication crisis.