Quizbot: exploring formative feedback with conversational interfaces

Abstract

Conversational interfaces (also called chatbots) have recently disrupted the Internet and opened up endless opportunities for assessment and learning. Formative feedback that provides learners with practical instructions for improvement is one of the challenging tasks in self-assessment settings and self-directed learning. This becomes even more challenging if a user’s personal information such as learning history and previous achievements cannot be exploited for data protection reasons or are simply not available. This study seeks to explore the opportunities of providing formative feedback in chatbot-based self-assessment. Two main challenges were faced: the limitations of the messenger as an interface that restricts visual representation of the quiz questions, and zero information about the user to generate adaptive feedback. Two types of feedback were investigated regarding their formative effect: immediate feedback, which was given after answering a question, and cumulative feedback detailing strengths and weaknesses of the user in each of the topics covered along with the directives for improvement. A chatbot called SQL Quizbot was deployed on Facebook Messenger for the purposes of this study (Try out the prototype at https://www.messenger.com/t/2076690849324267). A survey conducted to disclose users’ perception of the feedback reveals that more than 80% of the users find immediate feedback helpful. Overall this study shows that chatbots have a great potential as an aiding tool for e-learning systems to include an interactive component into feedback in order to increase user motivation and retention.