Machine Learning-Assisted Systematic Review: A Case Study in Learning Analytics

Abstract

Traditional systematic reviews, despite their high-quality evidence, are labor-intensive and error-prone, especially during the abstract screening phase. This paper investigates the application of machine learning-assisted systematic reviewing in the context of Learning Analytics (LA) in higher education. This study evaluates two approaches—ASReview, an active traditional machine learning tool, and GPT-4o, a large language model—to automate this process. By comparing key performance metrics such as sensitivity, specificity, accuracy, precision, and F1-score, we assess the effectiveness of these tools against traditional manual methods. Our findings demonstrate the potential of machine learning to enhance the efficiency and accuracy of systematic reviews in learning analytics.