The importance of pilot studies for gamified research: Pre-testing gamettes to study supply chain decisions

Abstract

Games and game-like methodologies are increasingly used as research environments, especially to conduct experiments to study decision-making. This growing trend in conducting gamified experiments is suggestive of the need for pilot studies and pre-testing the underlying games as well as experimental design. In this paper, we present an example of conducting a pilot study using Gamettes, a game-based methodology for collecting rich behavioral data on human decisions. Our results shed light on the efficacy of our experiment, how Gamettes' players interact with it in the way it is designed, and help identify issues that could bias the experiment and lead to a failed study. They further confirm the importance of conducting and reporting such pilot studies in gamified research.