Comparison of linear, computerized adaptive and multi stage adaptive versions of the mathematics assessment of Turkish Pupil Monitoring system

Abstract

The purpose of this study is to compare the results of computer based linear Turkish Pupil Monitoring System (TPMS) administrations with Computerized Adaptive Testing (CAT) and Multi Stage Adaptive Testing (MSAT) results in mathematics assessment. On the basis of the real data obtained from TPMS, different CAT scenarios were tested in post-hoc simulations with various starting rules, termination criteria, and different control strategies of CAT using either Maximum Likelihood (ML) or Weighted Maximum Likelihood (WML) estimation procedures. Results of the CAT study indicated that WML with easy initial item difficulty, fixed test reliability termination along with item exposure and content control strategies produced defensible results. Alternatively, a multi stage scenario was designed to compare the efficiency of CAT and MSAT. Examinees were administered a fixed subtest with 15 items followed by two subtests having ten items each. Using MSAT in TPMS seemed to be producing more valid results in terms of content sampling than CAT.