Pengembangan Chatbot Perpustakaan Berbasis Retrieval-Augmented Generation (RAG) pada Telegram

Abstract

The library of Sam Ratulangi University (UNSRAT) has a large collection of books and academic resources; however, users still experience difficulties in obtaining book availability information quickly and efficiently. This study aims to develop and evaluate a Telegram-based chatbot using a Retrieval-Augmented Generation (RAG) approach integrated with the GPT-4.1-mini Large Language Model (LLM). The system was developed using the Waterfall method and implemented through the n8n workflow automation platform by integrating Telegram Bot, MySQL, and Pinecone as a vector database. The chatbot applies a Text-to-SQL RAG mechanism, where user questions are converted into embeddings, matched with database context, and transformed into SQL queries limited to SELECT operations. System evaluation was conducted using Black Box Testing, User Acceptance Testing (UAT), and RAG evaluation metrics consisting of Answer Relevancy and Faithfulness. The results show that the chatbot successfully performs its main functions and achieved a UAT score of 82.3%, while Answer Relevancy and Faithfulness obtained scores of 100%. The developed system is capable of providing relevant and interactive information regarding library collections.