Automatic Subject Descriptions of Polish Library and Information Science Articles: A Comparison of DESCRIPTOR E-Service and GPT-4o

Abstract

This study compares the Annif-based DESCRIPTOR e-service and GPT-4o for the automatic indexing of Polish library and information science articles using titles and abstracts. DESCRIPTOR e-service scored precision 0.27, recall 0.49, F1 0.34, and mean average precision 0.67; GPT-4o scored 0.27, 0.48, 0.34, and 0.75, respectively. The global Jaccard index was 0.22. Manually created descriptions contained fewer irrelevant terms than those produced by either tool; however, GPT-4o outperformed DESCRIPTOR e-service on this metric. Both tools identified relevant terms that were absent from the manual descriptions. Overall performance is moderate, so further improvements are needed for reliable automatic indexing of Polish resources.