A multi-rater nonparametric test of agreement and corresponding agreement plot

Abstract

In this note we develop a new method for testing agreement in both the two-dimensional and p-dimensional settings. Our approach takes advantage of some readily available tests of uniformity found in most statistical software packages. In addition, we develop a corresponding agreement plot and define the agreement region. We compare our approach to the concordance correlation coefficient and the intraclass correlation in two dimensions. We conclude that our new test of agreement is more suited towards examining the specific hypothesis of perfect agreement between raters, devices or other such types of measures as defined by points falling precisely on the line of concordance in terms of power and type I error control. © 2009 Elsevier B.V. All rights reserved.