Advancing open science through research assessment reform? Content analysis of CoARA action plans

Abstract

Research assessment reform and open science are frequently presented as aligned reform movements within science, with assessment reform expected to incentivize open science practices. However, empirical evidence of this integration remains limited. We conducted a quantitative content analysis of open science terminology in action plans from Coalition for Advancing Research Assessment (CoARA) signatories—institutions publicly committed to reforming research assessment. Analysing 248 publicly available action plans, we coded mentions of open science terms across values, umbrella, and operational categories. Results reveal skewed distributions of open science terms: most terms appear in fewer than 20% of documents. Values terms substantially outnumber operational terms in top mentions, suggesting organizations currently emphasize aspirational principles over concrete practices. Open Access emerges as the by far the most widely mentioned operational practice, while key activities emphasized in policy frameworks (Open Evaluation, Open Methods, Open Software) receive minimal attention. Overall, our findings suggest open science integration into assessment reform remains in early developmental stages, characterized by values-heavy discourse, and gaps between policy aspirations and organizational commitments. Beyond substantive findings, this study demonstrates how terminological analysis can track science reform movement development and provide data for longitudinal monitoring of open science and its convergence with reforms of research assessment.