Adapting to adaptive testing

Abstract

Computer adaptive testing (CAT) is a relatively recent innovation in large scale testing programs, but has had very limited application in private industry. This paper describes the development of a CAT for use by a large insurance company in selecting computer programmer trainees. Incumbents provided the calibration and evaluation data. The CAT led to increased item security, but did not decrease required testing time. Further, the CAT was found to be similar to a conventional, fixed-item test in reliability and validity. In addition to actual test results, computer simulated test data were used in a more detailed evaluation of the CAT's effectiveness. The concluding discussion notes the advantages and disadvantages observed from the use of adaptive testing.