How to evaluate educational games with refugee children: Methodological aspects and lessons learned from eduapp4syria

Abstract

Educational game evaluation is a multidimensional and complex phenomenon. The growing interest in game-based learning (GBL) results in an increasing need to evaluate the effects of this approach, which requires appropriate methods, techniques, and principles that can be applied by the GBL community. This paper reflects on the methodological aspects of evaluating educational games with refugee children drawing on practical experience and evaluation studies conducted in the EduApp4Syria project. The paper gives an overview of the project and presents three field studies conducted, including the GBL evaluation methods used in the context of refugee children. The methods used included quasi-experimental design, mixed-method approach, observation with/without checklist, questionnaires, interviews, pre/post-test (using EGRA), screen recording, game-logs, and expert evaluation. The evaluations illustrate the application and assessment of these methods. This paper presents the findings and pitfalls related to the applicability of evaluation methods in various phases of the game development life cycle and methodological and practical challenges in conducting research and eliciting data in the context of evaluating educational games with refugee children. This article provides an up-to-date examination of both methodological challenges common to GBL evaluation and those unique to the user group of refugee children, culminating in guidance for researchers on methods and critical issues that need to be considered when designing research studies involving educational games and children. The paper assists researchers to critically reflect on these methodological issues and methods they use as they will have implications on the data obtained.