Validity and fairness: Towards a meaningful assessment in standardized writing tests

Abstract

Until recently, validity of standardized writing tests was mostly based on interrater reliability, writing tasks were not informed by theory and research on literacy, texts were assessed using proxy measures or analytical rubrics that compartmentalized writing traits, and, most importantly, contextual validation, fairness and consequences of uses of tests' results were not considered. This paper presents the development of a new writing assessment to improve the information on student's written performance available at the beginning of tertiary education in Chile. The new test includes prompts to write for academic purposes and social participation; provides details on the rhetorical situation, purposes, audiences, and assessment expectations; gives room for planning and revision; and rates texts according to a four-point holistic rubric that focuses on purpose achievement. Results show high levels of students' engagement and normal distribution of performance.