In classical statistics, the significance of comparisons (e.g., θ1 - θ2) is calibrated using the Type 1 error rate, relying on the assumption that the true difference is zero, which makes no sense in many applications. We set up a more relevant framework in which a true comparison can be positive or negative, and, based on the data, you can state "θ1 > θ2 with confidence," "θ2 > θ1 with confidence," or "no claim with confidence." We focus on the Type S (for sign) error, which occurs when you claim "θ1 > θ2 with confidence" when θ2 > θ1 (or vice-versa). We compute the Type S error rates for classical and Bayesian confidence statements and find that classical Type S error rates can be extremely high (up to 50%). Bayesian confidence statements are conservative, in the sense that claims based on 95% posterior intervals have Type S error rates between 0 and 2.5%. For multiple comparison situations, the conclusions are similar.
CITATION STYLE
Gelman, A., & Tuerlinckx, F. (2000). Type S error rates classical and Bayesian single and multiple compparison procedures. Computational Statistics, 15(3), 373–390. https://doi.org/10.1007/s001800000040
Mendeley helps you to discover research relevant for your work.