Comparing Three Groups

Abstract

For multiple comparisons in analysis of variance, the practitioners’ handbooks generally advocate standard methods such as Bonferroni, or an F-test followed by Tukey’s honest significant difference method. These methods are known to be suboptimal compared to closed testing procedures, but improved methods can be complex in the general multigroup set-up. In this note, we argue that the case of three-groups is special: with three groups, closed testing procedures are powerful and easy to use. We describe four different closed testing procedures specifically for the three-group set-up. The choice of method should be determined by assessing which of the comparisons are considered primary and which are secondary, as dictated by subject-matter considerations. We describe how all four methods can be used with any standard software.