Investigating the effect of artificial intelligence in education (AIEd) on learning achievement: A meta-analysis and research synthesis

Abstract

Scant information exists about how AI with its different technologies might affect learning achievement in different educational fields across different educational levels and geographical distributions of students. Closing this gap can therefore help stakeholders understand under which learning conditions artificial intelligence in education (AIEd) might work or not, hence achieving better learning achievement. To address this research gap, this study conducted a meta-analysis and research synthesis of the effects of AI application on students’ learning achievement. Additionally, this study conducted one step forward to analyze the field of education, level of education, learning mode, intervention duration, and geographical distribution as moderating variables of the effect of AIEd. The Hedges’ g was computed for the effect sizes, where 85 quantitative studies (N = 10,469 participants) were coded and analyzed. The results indicated that the total effect of AIEd on learning achievement is very large (g = 1.10, p < 0.001). Particularly, chatbots achieved a very large effect, while Intelligent Tutoring Systems (ITS) and personalized learning systems had large effects. The results also show that the AIEd effect is moderated by the field of education, level of education, learning mode, intervention duration, and geographical distribution of students. The findings of this study can be useful to both researchers and practitioners as they highlight how and when AIEd integration can be effective, hence being beneficial to enhance learning achievement.