Significance testing, p-values and the principle of total evidence

Abstract

The paper examines the claim that significance testing violates the Principle of Total Evidence (PTE). I argue that p-values violate PTE for two-sided tests but satisfy PTE for one-sided tests invoking a sufficient test statistic independent of the preferred theory of evidence. While the focus of the paper is to evaluate a particular claim about the relationship of significance testing and PTE, I clarify the reading of this methodological principle along the way.