ChatCRS: Incorporating External Knowledge and Goal Guidance for LLM-based Conversational Recommender Systems

Abstract

We enable large language models (LLMs) to efficiently use external knowledge and goal guidance in conversational recommender system (CRS) tasks. LLMs currently achieve limited effectiveness in domain-specific CRS tasks for 1) generating grounded responses with recommendation-oriented knowledge, or 2) proactively leading the conversations through different dialogue goals. We analyze these limitations through a comprehensive evaluation, showing the necessity of external knowledge and goal guidance which contribute significantly to the recommendation accuracy and language quality. This finding leads us to propose the ChatCRS framework, which decomposes the complex task of CRS into sub-tasks through the implementation of 1) a knowledge retrieval agent using a tool-augmented approach to reason over external knowledge bases, and 2) a goal-planning agent for dialogue goal prediction. By incorporating these inputs, LLMs proactively plan interactions and generate outputs with rich information. Experiments on two multi-goal CRS datasets reveal that ChatCRS sets new state-of-the-art performance, improving language quality of informativeness by 17% and proactivity by 27%, with a tenfold recommendation accuracy enhancement1