Assessing Learning from, with, and in Games Revisited: A Heuristic for Emerging Methods and Commercial Off-the-Shelf Games

Abstract

Technology systems and tools have evolved; current technologies are connected, pervasive, and smart. Digital systems like games have become more dynamic, emergent, and complex. In response to these advancements, considerable effort has been applied to understand how people learn from, with, and within game-based environments. Scientists continue to expand the perspectives and views on humans and their interactions with systems like video games. Further, the methods and tools available to frame research, capture and extract data from these environments, and contextualize the inferences from the patterns discovered among the data have also evolved. This chapter coalesces experience from research with video games informed by three key perspectives: (1) games are complex systems; (2) human-computer interaction is a viable framework to describe learning with and in these systems; and (3) process-oriented data extracted from games can be informed from an analytics perspective. Specifically, six principles are presented that may help researchers engage in studies that involve the process of learning. These emerged from several years of research and include several key questions associated with special considerations, challenges, or pitfalls when deciding on a system, research questions, framework, etc. Subsequently, the chapter offers a discussion of existing/ongoing research involving Bully, The Deed, and League of Legends, including (a) detailed descriptions of the games, (b) discussions of each relevant class of affordances (i.e., player, researcher, and developer), and (c) practical research examples, including the purpose, method, and strategies to analyze data. Finally, the ways the heuristic and its principles guided each of the research examples are discussed as a means to contextualize broader implications for learning, assessment, research, and design.