Can Sustainable Development Goal Classification Tools Be Trusted? Analysing the Spanish National Research Council Output on Gender Equality

Abstract

Evaluating research contributions to the Sustainable Development Goals (SDGs) remains challenging, particularly for SDG 5 (Gender Equality), which is cross-cutting, thematically complex and apparently underrepresented in scientific output. This study evaluates the abilities and limitations of automated classification systems (Web of Science, Scopus, OpenAlex and OSDG) in identifying SDG 5 related publications, through a case study of the Spanish National Research Council (CSIC) publications between 2018 and 2022. A mixed approach, combining bibliometric analysis and expert review, examined overlaps among sources, alignment with SDG 5 targets and thematic relevance. Results reveal very low agreement across tools, with only 35% of suggested publications validated as truly related. Most addressed only a few specific targets and semantic ambiguities and certain “two-sided” concepts/areas whose presence did not always indicate alignment with the goal. Findings highlight the need for hybrid approaches that integrate automated detection with expert qualitative evaluation to ensure rigorous SDG evaluation.