Measuring the Impact of Emerging Technologies in Education: A Pragmatic Approach

Abstract

The evaluation of emerging technologies is important for their impacts to be effectively integrated into learning and teaching settings to bring the best benefit to learners and teachers. Educators, learners, parents and policymakers alike, therefore, need reliable methodologies for evaluating the effectiveness of such emerging technologies. However, the impact evaluations of technology in education are challenging. This challenge is more significant for emerging technologies, as change is the essense of emerging educational technologies. Therefore, the value of traditional impact evaluations in education require being reconsidered within this context. Here, we present a pragmatic approach to measuring the impact of emerging technologies in education which focuses on the suitability of the proposed evaluation methods and the types of evidence rather than on the hierarchy of these methods and evidence types. The approach has two main steps. First one is the creation of a clear theory of change to identify outcome measure(s) and assumptions that are behind the expected impact of the emerging technology intervention. Secondly, the identification of the type of evidence and methods to generate it that are the most appropriate for the current innovation stage of the emerging technology.