Using a modified multidimensional priority index for item selection underwithin-item multidimensional computerized: Adaptive testing

Abstract

Computerized adaptive testing (CAT) not only enables efficient and precise ability estimation but also increases the security of testing materials since examinees are given different sets of items from a large item bank. The construction of assessments usually involves fulfilling a large number of non-statistical constraints, such as item exposure control and content balancing. To improve measurement precision, test security, and test validity, the priority index (PI) and multidimensional priority index (MPI) were proposed to monitor many constraints simultaneously for unidimensional and multidimensional CATs, respectively. Many educational and psychological tests are constructed under a multidimensional framework. Some of the items (multidimensional items) in a test are often intended to assess multiple latent traits. However, Yao’s MPI method was developed for a between-item multidimensional framework. When a within-item multidimensional test is assembled, a modifiedMPI algorithm is necessary. Therefore, the purposes of the study were to derive an algorithm for the modified MPI method for the withinitem multidimensional CATs and to investigate the efficiency of the modified MPI method through simulations.