A study of user experience in knowledge-based QA chatbot design

Abstract

A chatbot is a computer program which can interact with humans through natural language. In this paper, we introduced Jiao Xiao Tong, an interactive question answering chatbot designed in assisting students and faculties in Shanghai Jiao Tong University to accomplish various campus activities such as setting remainders, answering school trivia questions, route navigation and etc. A two-part experiment was conducted to evaluate two aspects of chatbot user experience. The results show an 6.7% increase in user satisfaction rate using the chatbot in route navigation tasks and interests towards a more interactive approach. The study also suggests that users expect action cues when interacting with a chatbot. In this paper, we described a solution of knowledge-based QA chatbot design in a university context and discussed what level of optimizations was achieved and what remains to be done. This pilot study provides opportunities for further improvement on chatbots design and application.