A Coefficient of Agreement for Nominal Scales

Abstract

A coefficient of interjudge agreement for nominal scales, formula-omitted, is presented. It is directly interpretable as the pro-portion of joint judgments in which there is agreement, after chance agreement is excluded. Its upper limit is +1.00, and its lower limit falls between zero and -1.00, depending on the distribution of judgments by the two judges. The maximum value which x can take for any given problem is given, and the implications of this value to the question of agreement discussed. An interesting characteristic of x is its identity with 0 in the dichotomous case when the judges give the same marginal distributions. Finally, its standard error and techniques for estimation and hypothesis testing are presented. © 1960, Sage Publications. All rights reserved.