New item-selection methods for balancing test efficiency against item-bank usage efficiency in CD-CAT

Abstract

Cognitive diagnostic computerized adaptive testing (CD-CAT) is a popular mode of online testing for cognitive diagnostic assessment (CDA). A key issue in CD-CAT programs is item-selection methods. Existing popular methods can achieve high measurement efficiencies but fail to yield balanced item-bank usage. Diagnostic tests often have low stakes, so item overexposure may not be a major concern. However, item underexposure leads to wasted time and money on item development, and high test overlap leads to intense practice effects, which in turn threaten test validity. The question is how to improve item-bank usage without sacrificing too much measurement precision (i.e., the correct recovery of knowledge states) in CD-CAT, which is the major purpose of this study. We have developed several item-selection methods that successfully meet this goal. In addition, we have investigated the Kullback–Leibler expected discrimination (KL-ED) method that considers only measurement precision except for item-bank usage.