Arguing Validity in Educational Assessment

Abstract

Interpreting test-scores in terms of whether examinees reach specific levels of achievement (e.g., below basic, basic, advanced), provides a means to assess and communicate whether educational goals are being reached and expectations are being met. Whether these interpretations of test-scores are informative, however, hinges on their validity. While validity plays an important role in educational assessment, it is rarely addressed in a systematic and comprehensive manner. Our aim is to detail a theoretical framework in which validation is considered in the context of practical test development. To this end, we apply Kane’s (Psychol Bull 112:527–535, 1992; Rev Edu Res 64:425–461, 1994) interpretive argument approach and Toulmin’s inference model (Kane, Lang Test 29:3–17, 2011; Toulmin, The uses of argument. Cambridge University Press, Cambridge) to the development of competence-based educational assessments and the interpretation of their results. A logical argument is presented to provide a theoretical framework for evaluating the rhetorical backing and empirical evidence supporting interpretations of educational assessment results. The discussion focusses on the role of standard setting procedures which define minimum passing scores on test-score scales in the evaluation of the validity of test-score interpretations.