Evaluation of strategic research programs: The case of Danish environmental research 1993-2002

Abstract

The paper reports the mid-term and final informetric evaluations of the Danish Strategic Environmental Research Program (named SMP). SMP consisted of nine virtual research centres during the period 1993-97. Citations are measured 1993-2002. Central indicators are: Centre Impact Factor (CIF) that sums up number of citations received by each centre's SCI-articles; centre Journal Impact Factor (JIF), which is a diachronic IF per journal volume publishing a centre article. Citation and publication data are obtained from the Dialog online version of SCI. Other indicators are the Danish and world domain impact in subject areas selected by the centres from the National Science Indicators. Top-ranked journal volumes used in SMP in terms of JIF scores were correlated with the corresponding articles' citation values. At the mid-term assessment the Pearson coefficient showed a strong correlation, which disappeared at the final evaluation. The publication behaviour varied substantially between centres. Although SMP as program did not make a strategic difference, measured as a CIF-score 10% higher than all other indicator values and in particular the Danish domain impact, four centres clearly did. USA was the most knowledge importing country. In wider perspective two-step evaluations of, in particular, cost-heavy strategic programmes have important implications for the continuation, volume and direction of research funding and activities.