Performance Evaluation of Keyword Detection for the Chatbot Model

Abstract

To enable enriched free-text human-computer conversations, keyword detection is an important component in chatbot models as it helps to identify specific keywords in user inputs that can trigger the chatbot to respond in a certain way. The performance of keyword detection thus depends on several factors such as the quality and quantity of training data, the selection of apt learning algorithms, and the tuning of various parameters. The performance evaluation of keyword detection for a chatbot involves taking into consideration, the specific requirements of the chatbot and the expected usage patterns of its users. To that end, this work investigates the keyword detection performance with limited vocabulary in the closed-domain chatbot model. A keyword reduction methodology is presented and experimental results on one public and one custom closed-domain datasets indicate about 4.6% improvement in F1-score and comparable performance respectively.