Measuring academic research impact: creating a citation profile using the conceptual framework for implementation fidelity as a case study

Abstract

The “citation score” remains the most commonly-used measure of academic impact, but is also viewed as practically and conceptually limited. The aim of this case study was to test the feasibility of creating a “citation profile” for a single, frequently-cited methods paper, the author’s own publication on the conceptual framework for implementation fidelity. This was a proof-of-concept study that involved an analysis of the citations of a single publication. This analysis involved identifying all citing publications and recording, not only how many times the key paper was cited within each citing publication, but also within which sections of that publication (e.g. Background, Methods, Results etc.). Level of impact could be categorised as high, moderate or low. The key paper had been cited more than 400 times and had a high impact in 25 % of publications based on citation frequency within publications, i.e. the key paper was cited three or more times; and a low impact in 58 % of citing publications, i.e. the key paper was cited just once. There were 41 “high impact” publications based on location of the citations, of which 35 (85 %) were also categorised as high impact by frequency. These results suggest that it is both possible and straightforward to categorise the level of impact of a key paper based on its “citation profile”, i.e., the frequency with which the paper is cited within citing publications, thus adding depth and value to the citation metric.