HelpAgent: Explainable Agent-based Helpdesk System

Abstract

As human-agent interactions become increasingly prevalent, designing systems that foster trust, transparency, and user autonomy is crucial. This paper introduces HelpAgent, an agent-based helpdesk system that can integrate Explainable Artificial Intelligence (XAI) to improve both user experience and operational efficiency. The proposed system utilizes a classifier to automate ticket categorization, offering users the option to either accept the agent's classification or override it based on personal judgment. A key innovation is the use of Large Language Models (LLMs) to transform complex SHapley Additive exPlanation (SHAP) results into non-expert-friendly narratives through LLMs, ensuring explanations are accessible to both expert and non-expert users. To evaluate system performance, we developed a classification model using multiple Machine Learning (ML) and Deep Learning (DL) architectures, with pre-trained models such as Large Language Model Meta AI (LlaMA) achieving the highest performance. User testing reveals that the majority of participants preferred the proposed system over traditional methods, citing improved usability and trust in the Artificial Intelligence (AI)-driven processes. This work demonstrates the potential of agent-based systems to streamline support workflows while enhancing user satisfaction through explainability and interaction flexibility.