Multidimensional Hybrid Computerized Adaptive Testing Based on Multidimensional Item Response Theory

Abstract

Computerized adaptive testing (CAT) and multistage adaptive testing (MST) are widely used to deliver assessment questions in the fields of psychometrics, educational measurement, and medical assessments. Hybrid computerized adaptive testing (HCAT), as a novel and flexible approach that incorporates both modular and adaptively-selected items, effectively integrates the CAT and MST, and inherits their respective strengths. Current HCAT focuses on unidimensional assessments, yet practical applications often require multidimensional assessments. Multidimensional item response theory (MIRT) models can provide accurate measurement of examinees’ multidimensional latent traits. Based on the MIRT models, this study proposes an innovative approach for constructing multidimensional hybrid computerized adaptive testing (MHCAT), aimed at better accommodating complex testing demands. Simulation studies were conducted to evaluate MHCAT using both dichotomous and polytomous items. Results indicated that, the fixed-length MHCAT achieved similar estimation accuracy to the fixed-length multidimensional CAT (MCAT), and the variable-length MHCAT had slightly higher estimation accuracy than the variable-length MCAT. Regarding item exposure control, both the fixed-length and variable-length MHCAT performed better than the MCAT. Empirical studies further validated the feasibility of MHCAT with several MIRT models. In summary, the proposed MHCAT presents promising performance in assessing examinees’ abilities while maintaining satisfactory item exposure control, providing a valuable approach for multidimensional assessments.