A Comparative Assessment of US and PH Learner Traversals and In-Game Observations within Minecraft

Abstract

What-If Hypothetical Implementations using Minecraft (WHIMC) is a set of Minecraft worlds that learners can explore to learn more about science, mathematics, engineering, and technology. Because Minecraft is open-ended by design, assessing whether students are learning is always a challenge. In addition to using methods such as post-tests and self-reports, how can we use in-game data to measure learning? In this paper, we analyze and compare American (US) and Filipino (PH) learner traversals and in-game observations against canonical answers from experts to determine the extent to which students achieved the desired learning outcomes and the ways in which outcomes varied. We grouped students into high- and low-performing categories based on an out-of-game post test. We found that high-performers tended to make more observations than low-performers. Observations of high-performers tended to align with canonical answers. Many PH students tended to be low-performers and tended not to make in-game observations. In contrast, even low-performing US students tended to record observations actively.