Critical evaluation of data requires rigorous but broadly based statistical inference

Abstract

The rampant misuse of the P value and of its stated meaning lead the American Statistical Association to comment, researchers often wish to turn a P value into a statement about the truth of a null hypothesis, or about the probability that random chance produced the observed data. The P value is neither. It is a statement about data in relation to a specified hypothetical explanation and is not a statement about the explanation itself.1 The American Statistical Association is not alone in its concern: a host of recent literature provides context for the desire to use statistical inference to reinforce rigor and reproducibility in scientific research.2-4 A central focus of this literature is the widely acknowledged and severe limitations we impose on ourselves with a blind and naive adherence to the exclusive use of P values for understanding significance of research findings.