AI-Enhanced Research Coaching for Graduate Education: A Mixed-Methods Evaluation of Learning Outcomes and Educational Design Effectiveness

Abstract

Background/purpose. This study addresses challenges in graduate research supervision by developing and evaluating an AI chatbot research coaching system designed through Design Thinking and Educational Design Research. The research aimed to examine system usability, perceived value, and alignment between user experiences and original design principles. Materials/methods. Thirty graduate students from Learning Management Innovation programs at a Thai university participated in a mixed-methods evaluation. The system, grounded in the GROW coaching model and delivered via LINE and Dialogflow, provided structured guidance, templates, and automated feedback. Data collection included Likert-scale questionnaires, open-ended reflections, and focus group discussions, which the researchers analyzed using a convergent mixed-methods approach. Results. The system achieved high satisfaction (M = 4.52) and strong emotional engagement across diverse levels of digital literacy. Qualitative analysis revealed six themes, including enhanced understanding of research, increased confidence, and perceived social presence. Participants developed unexpected emotional attachments, describing the system as a "thinking partner." A strong alignment emerged between user feedback and design principles, yielding notable outcomes in conceptual understanding, skill application, and milestone completion. Conclusion. Results demonstrate that emotionally intelligent AI coaching systems can effectively support graduate research development by addressing both cognitive and affective learning dimensions. The study provides a validated framework for developing culturally sensitive, user-centered educational technologies in resource-constrained contexts.