New method for assessment of raters agreement based on fuzzy similarity

Abstract

In measurement systems one of the components affecting its variation is a human factor. Man - as a process operator - measures or rates the product. Since his decisions may have significant impact on the customer satisfaction, his reliability and usefulness for the measuring tasks should be evaluated. This article describes authors proposal for a new measurement system analysis methodology for unmeasurable features. In this method many features of the product are rated during one study, and the value of them can be expressed in a nominal or an ordinal (with imprecise data) measurement scale, and each of the features can be weighted (as less or more important from the customer point of view). The goal of the methodology is to gain information about raters’ ability to define the value of each feature in relation to the specification and customer requirements. The authors propose to use a similarity measure for rates assignment of product features values. The method is based on the new fuzzy similarity coefficient SC, and the level of agreement of the final decisions of the raters is analyzed based on Gwet’s AC1 coefficient.