Pretest Item Calibration in Computerized Multistage Adaptive Testing

Abstract

The purpose of this study was to compare calibration and linking methods for placing pretest item parameter estimates on the item pool scale in a 1-3 computerized multistage adaptive testing design in terms of item parameter recovery. Two models were used: embedded-section, in which pretest items were administered within a separate module, and embedded-items, in which pretest items were distributed across operational modules. The calibration methods were separate calibration with linking (SC) and fixed calibration (FC) with three parallel approaches under each (FC-1 and SC-1; FC-2 and SC-2; and FC-3 and SC-3). The FC-1 and SC-1 used only operational items in the routing module to link pretest items. The FC-2 and SC-2 also used only operational items in the routing module for linking, but in addition, the operational items in second stage modules were freely estimated. The FC-3 and SC-3 used operational items in all modules to link pretest items. The third calibration approach (i.e., FC-3 and SC-3) yielded the best results. For all three approaches, SC outperformed FC in all study conditions which were module length, sample size and examinee distributions.