Usability and User Experience of a Chat Application with Integrated Educational Chatbot Functionalities

Abstract

In recent years chatbots have found their use in education, especially in higher education (HE), but are they really serving a purpose and helping both students and faculty staff? Since bad experiences can discourage potential users, chatbots have to be carefully designed to be useful and to offer the best possible experience to its users. To explore this premise, in this paper we will tackle one aspect of chatbot design through evaluation of usability and user experience of a chat application Differ, which has integrated chatbot functionalities. For the duration of one semester at the Faculty of Organization and Informatics’ students used Differ on several courses. Differ was introduced as a tool to facilitate communication related to courses and students’ obligations, between students and educators by initiating communication through a Differ’s chatbot called Buddy Bot. Differ was also seen as a tool to enhance students’ connections and social life through creation of private student communities. After the end of semester, students’ opinions about Differ were evaluated through means of UEQ and SUS questionnaires which are frequently used for user experience and usability evaluation. Results of the study showed that Differ’s perceived usability and user experience are slightly above average, but with a lot of room for improvements, especially in the context of novelty, stimulation and chatbot functionalities.