Overton: A bibliometric database of policy document citations

Abstract

This paper presents an analysis of the Overton policy document database, describing the makeup of materials indexed and the nature in which they cite academic literature. We report on various aspects of the data, including growth, geographic spread, language representation, the range of policy source types included, and the availability of citation links in documents. Longitudinal analysis over established journal category schemes is used to reveal the scale and disciplinary focus of citations and determine the feasibility of developing field-normalized citation indicators. To corroborate the data indexed, we also examine how well self-reported funding outcomes collected by UK funders correspond to data indexed in the Overton database. Finally, to test the data in an experimental setting, we assess whether peer-review assessment of impact as measured by the UK Research Excellence Framework (REF) 2014 correlates with derived policy citation metrics. Our findings show that for some research topics, such as health, economics, social care, and the environment, Overton contains a core set of policy documents with sufficient citation linkage to academic literature to support various citation analyses that may be informative in research evaluation, impact assessment, and policy review.