The stability of test design: measuring differences in performance across several administrations of an academic literacy test

Abstract

This study explores the stability or consistency across several administrations of a test designed to determine academic literacy levels. The reliability of two versions (one English, the other Afrikaans) of such a test used for placement purposes at three South African universities will first be analysed. Secondly, the number of potential misclassifications on the test is assessed (ie the extent to which the test does not measure fairly). Thirdly, the differences among the results of the various administrations of the test are explored, with a view to es-tablishing whether such differences are both significant and relevant.