From framework to survey data: Inside the PISA assessment process

Abstract

This chapter provides an overview of quality assurance mechanisms that have been put into place by the international contractor responsible for implementing the PISA survey. These quality assurance mechanisms aim to ensure the fitness for purpose of the PISA data, derived from over 60 different countries and from students instructed in over 40 different languages in a wide array of schools from education systems that vary quite considerably. The mechanisms reviewed include the frameworks that drive the content of the PISA survey instruments, the processes followed in test item development, student sampling procedures, the mechanisms designed to guarantee comparability of the different versions of test instruments that go into the field in participating countries, steps to ensure test administration procedures are common across all test administration centres, the mechanisms associated the processing and scoring of student responses to the test questions, and processes related to capturing, processing and analysing PISA data. This chapter also brings together into an accessible and consolidated form, information that has been published in a variety of other documents, such as in PISA operational manuals and technical reports. However, the chapter also explains the significance of these processes and the reasons for the decisions, and highlights how they are implemented for mathematics.