Study Design and Evolution, and the Imperatives of Reliability and Validity

Abstract

During the six decades since the International Association for the Evaluation of Educational Achievement (IEA) began its program of studies, an ever-growing political impetus worldwide for improved educational provision has stimulated countries’ willingness to participate in international large-scale comparative assessments of learning outcomes. However, research within the complex multinational context that IEA operates in has resulted in significant methodological and technological challenges for the researchers and practitioners endeavoring to meet the goals of IEA studies. Such studies must satisfy the twin imperatives of validity and reliability, rarely an easy task given the multiple and diverse cultures, languages, scripts, educational structures, educational histories, and traditions of the countries and jurisdictions that participate. An appreciation of IEA’s underlying assessment philosophy is fundamental to understanding the Association’s assessment goals and the key design features of its studies, and what these mean with respect to ensuring that the studies satisfactorily address the demands of reliability and validity.