Unveiling the core constructs: A statistical approach to evaluating user experience with Chatbots in higher education (A case study from a university in Ecuador)

Abstract

Introduction: Chatbots are increasingly utilized in education, offering real-time, personalized communication. While research has explored technical aspects of chatbots, user experience remains under-investigated. This study examines a model for evaluating user experience and satisfaction with chatbots in higher education. Methodology: A four-factor model (information quality, system quality, chatbot experience, user satisfaction) was proposed based on prior research. An alternative two-factor model emerged through exploratory factor analysis, focusing on “Chatbot Response Quality” and “User Experience and Satisfaction with the Chatbot.” Surveys were distributed to students and faculty at a university in Ecuador to collect data. Confirmatory factor analysis validated both models. Results: The two-factor model explained a significantly greater proportion of the data’s variance (55.2%) compared to the four-factor model (46.4%). Conclusion: This study suggests that a simpler model focusing on chatbot response quality and user experience is more effective for evaluating chatbots in education. Future research can explore methods to optimize these factors and improve the learning experience for students.