Comparative analysis of large language models’ performance in book classification tasks using Library of Congress Classification system

Abstract

The advancement of artificial intelligence (AI) shows the potential to facilitate information organisation, providing opportunities for classification applications in libraries. While previous studies have investigated the potential usage of AI in libraries, its real-world integration into specific tasks like cataloguing is limited. In our study, we employ large language models (LLMs) to develop classifiers for book classification using the Library of Congress Classification system. We experiment with different input data, data sizes and models to identify effective settings for assisting cataloguing using LLMs. In addition, we evaluate the model performance at different levels of Library of Congress Classification (LCC) system class granularity. The results show that Llama3 outperforms the other five models in our experiment. More diverse input data may contribute to enhancing model performance. However, LLMs demonstrate limitations in processing long text and handling more granular categories.