A new Approach for Qualitative Proficiency Testing Assessment Based on Homogeneity Proportion Tests

Abstract

In qualitative proficiency testing schemes, the participants’ results are categorical classifications of test items. In this case, the goal is to assess the percentage of test items correctly classified as the provider’s expected result. The ISO 13528 international standard does not provide a detailed description of statistical methods for categorical data analysis. This work proposes a novel approach to assessing laboratories’ performance in qualitative proficiency testing based on homogeneity proportion tests. Overall and pairwise homogeneity proportion tests identify atypical percentage among the participating laboratories. In this approach, the outlying percentage is unacceptable. This methodology was compared with three agreement measures: Cohen’s Kappa, Krippendorff’s alpha, and Gwet’s gamma. Preliminary results indicate that the number of test items sent to each participant may influence the performance evaluation of the participants.