Towards Human-Centered RAG: A Study of AI-Supported Testing Practices in Italian Public Administration

Abstract

This study examines the integration of a Retrieval-Augmented Generation (RAG)-based AI testing assistant within an Italian public administration, focusing on its practical benefits and challenges. Qualitative feedback from 15 software testers reveals that while the assistant improves testing efficiency and supports knowledge retrieval, challenges regarding accuracy, consistency, and transparency reduce user trust and effectiveness. Participants highlighted the need for enhanced data curation, improved interaction, and finer-grained control over the system, indicating areas for future improvements to the assistant and its surrounding workflows. The findings emphasize the importance of adopting a human-centered approach in the design and integration of AI solutions in the public sector, offering valuable insights for the ongoing development of human-centered RAG research.