Pilot testing of the Research Impact Assessment Framework

Abstract

Traditional academic impact frameworks and metrics have been criticised, because they fail to assess the ‘real-world’ value of research. The Research Impact Assessment Framework was developed to complement and extend existing impact assessment frameworks by focusing on health system needs and priorities. This paper reports on piloting of the Research Impact Assessment Framework for feasibility and utility with researchers working in health and research organisations in Sydney, Australia. The Framework’s Research Environment domain was assessed through a bespoke employee survey, and the Alignment and Influence of Research domain through independent review of case studies. Using synthetic data, the outputs from the two domains were combined to compare institutions. The tools tested in this pilot are feasible to use in assessing an organisation’s capacity to produce impactful research that is aligned with identified health, social and economic priorities. The case study reviews illustrated the challenge of determining potential versus realised impact. This study identified that implementation of the Research Impact Assessment Framework will be enabled by use of existing routinely collected data, such as annual employee surveys, and the use of artificial intelligence tools to reduce administrative burden and ensure consistency of outputs.