Research evaluation reform and the heterogeneity of researchers' metric-wiseness

Abstract

Purpose: We aimed to measure the variation in researchers' knowledge and attitudes towards bibliometric indicators. The focus is on mapping the heterogeneity of this metric-wiseness within and between disciplines. Design/methodology/approach: An exploratory survey is administered to researchers at the Sapienza University of Rome, one of Europe's oldest and largest generalist universities. To measure metric-wiseness, we use attitude statements that are evaluated by a 5-point Likert scale. Moreover, we analyze documents of recent initiatives on assessment reform to shed light on how researchers' heterogeneous attitudes regarding and knowledge of bibliometric indicators are taken into account. Findings: We found great heterogeneity in researchers' metric-wiseness across scientific disciplines. In addition, within each discipline, we observed both supporters and critics of bibliometric indicators. From the document analysis, we found no reference to individual heterogeneity concerning researchers' metric wiseness. Research limitations: We used a self-selected sample of researchers from one Italian university as an exploratory case. Further research is needed to check the generalizability of our findings. Practical implications: To gain sufficient support for research evaluation practices, it is key to consider researchers' diverse attitudes towards indicators. Originality/value: We contribute to the current debate on reforming research assessment by providing a novel empirical measurement of researchers' knowledge and attitudes towards bibliometric indicators and discussing the importance of the obtained results for improving current research evaluation systems.