The implications of understanding that PISA is simply another standardized achievement test

Abstract

It occurred to me one day that despite all the excitement, and both the satisfaction and handwringing engaged in by some nations after scores are released, that the Program for International Student Assessment, PISA, is merely another Standardized Achievement Test. Almost all Standardized Achievement Tests (SATs) try to adhere to certain principles of design, have similar correlates, and have similar limits on the interpretations of the results obtained. Neither the popular press nor most politicians ordinarily understand these realities and their implications. Opinion makers are unaware that many of the Standardized Achievement Tests we commonly use do not have the powers attributed to them. It is not far from the truth to call the scores derived from some of these assessments “talismanic” (Haney et al. 1987). That is, for many people, test scores have special powers, particularly of prophesy, a bit like the Kabbalah of the middle ages. Scores from SATs are a part of the metrification associated with the modern world, no doubt aided by a global market place in which business leaders and technologists, instead of humanists and educators, have garnered political power. Contributing to this trend toward metrification has been the ascendance of economics as an influential discipline throughout the world (Lingard et al. 2015). But in my opinion, economists, journalists, and politicians too often seek in metrics powers that are more illusionary than they are real.