Generative AI and multimodal data for educational feedback: Insights from embodied math learning

Abstract

This study explores the role of generative AI (GenAI) in providing formative feedback in children's digital learning experiences, specifically in the context of mathematics education. Using multimodal data, the research compares AI-generated feedback with feedback from human instructors, focusing on its impact on children's learning outcomes. Children engaged with a digital body-scale number line to learn addition and subtraction of positive and negative integers through embodied interaction. The study followed a between-group design, with one group receiving feedback from a human instructor and the other from GenAI. Eye-tracking data and system logs were used to evaluate student's information processing behaviour and cognitive load. The results revealed that while task-based performance did not differ significantly between conditions, the GenAI feedback condition demonstrated lower cognitive load and students show different visual information processing strategies among the two conditions. The findings provide empirical support for the potential of GenAI to complement traditional teaching by providing structured and adaptive feedback that supports efficient learning. The study underscores the importance of hybrid intelligence approaches that integrate human and AI feedback to enhance learning through synergistic feedback. This research offers valuable insights for educators, developers and researchers aiming to design hybrid AI-human educational environments that promote effective learning outcomes. Practitioner notes What is already known about this topic? Embodied learning approaches have been shown to facilitate deeper cognitive processing by engaging students physically with learning materials, which is especially beneficial in abstract subjects like mathematics. GenAI has the potential to enhance educational experiences through personalized feedback, making it crucial for fostering student understanding and engagement. Previous research indicates that hybrid intelligence that combines AI with human instructors can contribute to improved educational outcomes. What this paper adds? This study empirically examines the effectiveness of GenAI-generated feedback when compared to human instructor feedback in the context of a multisensory environment (MSE) for math learning. Findings from system logs and eye-tracking analysis reveal that GenAI feedback can support learning effectively, particularly in helping students manage their cognitive load. The research uncovers that GenAI and teacher feedback lead to different information processing strategies. These findings provide actionable insights into how feedback modality influences cognitive engagement. Implications for practice and/or policy The integration of GenAI into educational settings presents an opportunity to enhance traditional teaching methods, enabling an adaptive learning environment that leverages the strengths of both AI and human feedback. Future educational practices should explore hybrid models that incorporate both AI and human feedback to create inclusive and effective learning experiences, adapting to the diverse needs of learners. Policymakers should establish guidelines and frameworks to facilitate the ethical and equitable adoption of GenAI technologies for learning. This includes addressing issues of trust, transparency and accessibility to ensure that GenAI systems are effectively supporting, rather than replacing, human instructors.