Qualifying the performance evaluation of Big Science beyond productivity, impact and costs

Abstract

The use of quantitative performance measures to evaluate the productivity, impact and quality of research has spread to almost all parts of public R&D systems, including Big Science where traditional measures of technical reliability of instruments and user oversubscription have been joined by publication counts to assess scientific productivity. But such performance assessment has been shown to lead to absurdities, as the calculated average cost of single journal publications easily may reach hundreds of millions of dollars. In this article, the issue of productivity and impact is therefore further qualified by the use of additional measures such as the immediacy index as well as network analysis to evaluate qualitative aspects of the impact of contemporary Big Science labs. Connecting to previous work within what has been called “facilitymetrics”, the article continues the search for relevant bibliometric measures of the performance of Big Science labs with the use of a case study of a recently opened facility that is advertised as contributing to “breakthrough” research, by using several more measures and thus qualifying the topic of performance evaluation in contemporary Big Science beyond simple counts of publications, citations, and costs.