A review of South Africa's national research foundation's ratings methodology from a social science perspective

Abstract

One of South Africa's National Research Foundation's (NRF) activities is to award ratings to academics who apply according to predefined categories. Explicitly or not, these ratings are part of submissions academics make for promotions and for employment in South African universities. As such, methodological assessment of the validity of this system is important. This paper seeks to conceptually evaluate certain characteristics of this system against certain general principles of reliability and validity. On the basis of the results of this evaluation, it is argued that assumptions that the NRF rating system is always valid or reliable as a differentiator of individual academics cannot be made unconditionally. Using Management Science as an example of a social science field that draws from multidisciplinary theoretical and methodological frameworks, this paper identifies certain validity issues associated with the current NRF rating system, and makes recommendations for improvements. Significance: •Certain validity issues are highlighted and arguments are made to improve the methodology used by the NRF to rate researchers. •Issues related to multidisciplinarity and mode two knowledge production are considered. •Technological advances that have made it possible for scientific measurement of research productivity and impact are discussed. •Problems with subjective methodologies are identified, together with their ethical consequences.