Zastosowanie współczynnika konkordancji w pomiarze zgodności ocen ekspertów

Abstract

In this article the problem of the measurement of agreement among experts is discussed. There are several tools to verify whether opinions expressed by an ordinal scale are reliable. If the objects are ranked, one of correlation coefficients is chosen. At the beginning, the case of m > 2 experts ranking n objects is presented; first, using untied ranks and second, tied ranks. To this end Kendall’s coefficient of concordance is applied. The method of examination of the statistical significance is also presented. Additionally the issue of the interpretation of the concordance coefficient in reference to the average of Spearman’s r coefficients is raised. The paper provides an example of using concordance coefficient when ranks are tied and discusses limitations of using rank correlation methods as a tool for evaluating the degree of agreement among experts’ opinions.