Domain-specific Embeddings for Question-Answering Systems: FAQs for Health Coaching

Abstract

FAQs are widely used to respond to users' knowledge needs within knowledge domains. While LLM might be a promising way to address user questions, they are still prone to hallucinations, i.e., inaccurate or wrong responses, which, can, inter alia, lead to massive problems, including, but not limited to, ethical issues. As a part of the healthcare coach chat-bot for young Nigerian HIV clients, the need to meet their information needs through FAQs is one of the main coaching requirements. In this paper, we explore if domain knowledge in HIV FAQs can be represented as text embeddings to retrieve similar questions matching user queries, thus improving the understanding of the chatbot and the satisfaction of the users. Specifically, we describe our approach to developing an FAQ chatbot for the domain of HIV. We used a predefined FAQ question-answer knowledge base in English and Pidgin co-created by HIV clients and experts from Nigeria and Switzerland. The results of the post-engagement survey show that the chatbot mostly understood the user's questions and could identify relevant matching questions and retrieve an appropriate response.