Measuring nominal scale agreement among many raters

Abstract

Introduced the statistic kappa to measure nominal scale agreement between a fixed pair of raters. Kappa was generalized to the case where each of a sample of 30 patients was rated on a nominal scale by the same number of psychiatrist raters (n = 6), but where the raters rating 1 s were not necessarily the same as those rating another. Large sample standard errors were derived. (PsycINFO Database Record (c) 2006 APA, all rights reserved). © 1971 American Psychological Association.