DERIVING COMPARABLE SCORES FOR COMPUTER ADAPTIVE AND CONVENTIONAL TESTS: AN EXAMPLE USING THE SAT1,2

Abstract

Procedures used to establish the comparability of scores derived from the College Board Admissions Testing Program (ATP) computer adaptive SAT prototype and the paper‐and‐pencil SAT are described in this report. Both the prototype, which is made up of Verbal and Mathematical computer adaptive tests (CATs), and a form of the paper‐and‐pencil test were administered to just greater than 500 examinees using a random groups counterbalanced design. Both linear and equipercentile procedures were used for equating in each of the separate testing orders (paper‐and‐pencil then CAT or CAT then paper‐and‐pencil). Data were not pooled across the orders because the groups were not randomly equivalent due to administrative problems. The linear procedure was chosen for each test (Verbal or Mathematical) for each order, and results from the two orders were averaged. The final Verbal and Mathematical CAT conversions were quite similar to the paper‐and‐pencil conversions, although the two conversions for Verbal and two conversions for Mathematical did differ by as much as 20 scaled score points in certain regions of the scale.