Where Is the Learning in Learning Analytics?: A Systematic Literature Review to Identify Measures of Affected Learning

Abstract

Learning analytics is the analysis and visualization of student data with the purpose of improving education. Literature reporting on measures of the effects of data-driven pedagogical interventions on learning and the environment in which this takes place, allows us to assess in what way learning analytics actually improves learning. We conducted a systematic literature review aimed at identifying such measures of data-driven improvement. A review of 1034 papers yielded 38 key studies, which were thoroughly analyzed on aspects like objective, affected learning and their operationalization (measures). Based on prevalent learning theories, we synthesized a classification scheme comprised of four categories: learning process, student performance, learning environment, and departmental performance. Most of the analyzed studies relate to either student performance or learning process. Based on the results, we recommend to make deliberate decisions on the (multiple) aspects of learning one tries to improve by the application of learning analytics. Our classification scheme with examples of measures may help both academics and practitioners doing so, as it allows for structured positioning of learning analytics benefits.