Learning analytics driven improvements in learning design in higher education: A systematic literature review

Abstract

Background: Driven by the ongoing need to provide high-quality learning and teaching, universities recently have shown an increased interest in using learning analytics (LA) for improving learning design (LD). However, the evidence of such improvements is scarce, and the maturity of such research is unclear. Objectives: This study is aimed to evaluate the maturity of research discussing LA-driven LD improvements in higher education. Methods: The systematic review analyses 49 empirical papers, assesses their quality and suggests further research directions. The review elaborates on methodological (research questions, strategy and methods, LA-LD integration theoretical backgrounds) and substantial (LA-driven LD improvements, types of data used, LA software development) features of the papers. Results and Conclusions: The findings demonstrated the lack of theoretical alignment between LA and LD, with research tending towards user experience studies. The most frequently used research strategy was a case study; experiments were very rare. Researchers predominantly used parsing for collecting data and AI methods for analysing it; mostly used data types related to registering learners' engagement with learning activities as well as resources and tools provided in digital learning environments. Takeaways: The research area discussing LA-driven LD improvements still has a way to go before attaining the level of full maturity. Only a third of the papers reported actual LA-driven LD improvements; moreover, only three papers measured their effectiveness. The presented LA software was mostly at the beta or implementation stages and did not assess the impact of using this software.