An anatomy of the academic ranking of world universities (Shanghai ranking)

Abstract

International academic rankings of research universities are widely applied and heavily criticised. Amongst the many international rankings, the Shanghai ranking has been particularly influential. Although this ranking’s primary data are generally accessible and its methods are published in outline format, it does not follow that its outputs are predictable or straightforward. In practice, the annual and time series Shanghai rankings rely on data and rules that are complex, variable, and not fully revealed. Patterns and changes in the ranking may be misinterpreted as intrinsic properties of institutions or systems when they are actually beyond the influence of any university or nation. This article dissects the rules that connect raw institutional data to the published ranking, using the 2020 edition as a reference. Analysing an ARWU review of ranking changes over 2004–2016, we show how exogenous or methodological changes have often driven changes in ranking. Stakeholders can be misled if they believe that changes are intrinsic to institutions’ performance. We hope to inform and warn the media, governments, and institutions about the merits and risks of using the Shanghai ranking to evaluate relative institutional performance and its evolution.