What Is In a (Non-) Significant Finding? Moving Beyond False Dichotomies

Abstract

One of the most problematic aspect of p-values is the tendency to dichotomize, whereby statistical significance is equated with scientifically worthwhile and insignificance with worthless findings. This article outlines two approaches that move beyond such dichotomies: minimum Bayes factors and the analysis of credibility. Their use is illustrated through a simple example. The examples show that significant findings may not always be as telling as statistical significance would suggest, whereas non-significant findings may contain considerable evidence about effects.