Classification consistency and accuracy for complex assessments under the compound multinomial model

Abstract

For a test that consists of dichotomously scored items, several approaches have been reported in the literature for estimating classification consistency and accuracy indices based on a single administration of a test. Classification consistency and accuracy have not been studied much, however, for ''complex'' assessments-for example, those that involve polytomously scored items or mixtures of different types of items. This article describes procedures for estimating various types of single-administration classification consistency and accuracy indices using the multinomial and compound multinomial models for individuals and groups who take a single administration of a complex assessment. An alternative procedure using the bootstrap is also proposed. These procedures are illustrated using real data sets obtained from tests consisting of both polytomous and dichotomous items. © 2009 SAGE Publications.