Satisfaction is insufficient: Insights from a randomized, controlled trial of a marketing simulation game

Abstract

Background: The development and promotion of educational games are still outpacing knowledge of these games' effects, raising calls for evidence of benefits and challenges. Studies suggest that students and teachers like games, but the payoff of the investment in terms of increased motivation and achievement remains unclear. Objectives: This study investigates the pure effect of a marketing simulation game on motivation, perceived learning and achievement, above and beyond regular student-active instruction. Methods: We applied a randomized, controlled experiment in a marketing course in upper-secondary schools (Nclasses = 22; Nstudents = 433) comparing a collaborative–competitive marketing simulation game with regular, case-based, student-active instruction on three groups of outcome measures: motivation, perceived ability, and achievement. Additionally, students and teachers provided quantitative and qualitative feedback on game experiences. Results and Conclusions: We showcase the importance of a robust study design with valid compound instruments. Moreover, investigations of the game implementation and experiences reveal insights about intervention timing, differential negative consequences by gender and need for reflection opportunities. We find no clear evidence of positive or negative effects of the game, despite students' and teachers' satisfaction. Implications: Beyond the effect evaluation, we offer recommendations to researchers and developers of educational games about scaffolding, timing and teacher competence building.