What Can Altmetric.com Tell Us About Policy Citations of Research? An Analysis of Altmetric.com Data for Research Articles from the University of Sheffield

Abstract

Background: There is a growing interest in using and analysing Altmetrics data for quantifying the impact of research, especially societal impact (Haunschild & Bornmann 2017, Thelwall et al 2016, Bornmann 2014). This study therefore aimed to explore the value of Altmetrics for assessing policy impact by analysing research articles from a single centre (the University of Sheffield) that were identified by Altmetrics as being cited in at least one policy document. Method: This study has only included published research articles from authors at the University of Sheffield and indexed in the Altmetric.com database. Altmetric data on policy impact were sourced from Altmetric.com following a data request and included citations up until February 2017. Supplementary Altmetric data including news media, blogs, Mendeley saves and Wikipedia citations were also gathered. Results: Altmetrics did enable an assessment of whether a relevant research article had had an impact on policy documents and, if so, how many such documents. In total, 1463 pieces of published research from authors at the University of Sheffield were found to be cited in between 1 and 13 policy documents. Twenty-one research articles were listed as being cited in five or more policy documents; 21 in four policy documents; 50 in three documents; 186 in two documents; and 1185 in one document. Of those 1463 outputs, 1449 were journal articles, 13 were books and one was a book chapter, and the time lag from publication to citation in policy documents ranged from 3 months to 31 years. Analysis of the 92 research articles cited in three or more policy documents indicated that the research topics with the greatest policy impact were medicine, dentistry and health, followed by social science and pure science. The Altmetrics data enabled an in-depth assessment of the 21 research articles cited in five or more policy documents. Conclusion. Within the limitations of the current text-mining system, Altmetrics can offer important and highly-accessible data on the policy impact of an organization’s published research articles.