March 14, 2007 17:14 WSPC/INSTRUCTION FILE SETN��Journal��v2 International Journal on Artificial Intelligence Tools c World Scientific Publishing Company MODELING AND AUGMENTING GAME ENTERTAINMENT THROUGH CHALLENGE AND CURIOSITY GEORGIOS N. YANNAKAKIS Maersk Mc-Kinney Moller Institute University of Southern Denmark, Campusvej 55 Odense M, 5230, Denmark georgios@mmmi.sdu.dk JOHN HALLAM Maersk Mc-Kinney Moller Institute University of Southern Denmark, Campusvej 55 Odense M, 5230, Denmark john@mmmi.sdu.dk Received (Day Month Year) Revised (Day Month Year) Accepted (Day Month Year) This paper presents quantitative measurements/metrics of qualitative entertainment features within computer game environments and proposes artificial intelligence (AI) techniques for optimizing enter- tainment in such interactive systems. A human-verified metric of interest (i.e. player entertainment in real-time) for predator/prey games and a neuro-evolution on-line learning (i.e. during play) approach have already been reported in the literature to serve this purpose. In this paper, an alternative quan- titative approach to entertainment modeling based on psychological studies in the field of computer games is introduced and a comparative study of the two approaches is presented. Feedforward neu- ral networks (NNs) and fuzzy-NNs are used to model player satisfaction (interest) in real-time and investigate quantitatively how the qualitative factors of challenge and curiosity contribute to human entertainment. We demonstrate that appropriate non-extreme levels of challenge and curiosity gener- ate high values of entertainment and we project the extensibility of the approach to other genres of digital entertainment (e.g. mixed-reality interactive playgrounds). Keywords: Entertainment modeling computer games mixed-reality games artificial neural networks. 1. Introduction Computer games, as examples of human-computer interactive systems, provide an ideal environment for research in AI, because they are based on simulations of highly complex and dynamic multi-agent worlds.1,2,3 Moreover, such systems offer a promising ground for cognitive modeling since they embed rich forms of interactivity between humans and non- player characters (NPCs).4 Being able to capture quantitatively the level of user (gamer) engagement or satisfaction in real-time can grant insights to the appropriate AI method- ology for enhancing the quality of playing experience and furthermore be used to adjust digital entertainment environments according to individual user preferences.5 1

March 14, 2007 17:14 WSPC/INSTRUCTION FILE SETN��Journal��v2 2 Georgios N. Yannakakis and John Hallam An endeavor on capturing player satisfaction during gameplay and providing quantita- tive measurements of entertainment in real-time is presented in this article. The principal goal in the reported work is to construct a user model of a class of game playing experi- ence. Specifically, the aim is that the model can predict the answers to which variants of a given game are more or less ���fun.��� This approach is referred to as Entertainment Mod- eling. Herein, entertainment is defined qualitatively primarily as the level of satisfaction generated by the real-time player-game opponent interaction ��� by ���opponent��� we mean any controllable interactive feature of the game. According to this definition, a game is pri- marily a learning process and the level of entertainment is kept high when game opponents enable new learning patterns (���not too easy a game���) for the player that can be perceived and learned by the player (���not too difficult a game���).6,7 On the same basis, according to Kapoor et al. learning is highly correlated to interest, curiosity and intrigue perceived within the axis of emotions varying from boredom to fascination.8 The collection of these emotions is defined as entertainment (or ���fun���) in this article. Entertainment capture in this paper is achieved by following the theoretical principles of Malone���s intrinsic qualitative factors for engaging gameplay,9 namely challenge (i.e. ���provide a goal whose attainment is uncertain���), curiosity (i.e. ���what will happen next in the game?���) and fantasy (i.e. ���show or evoke images of physical objects or social situa- tions not actually present���) and driven by the basic concepts of the theory of flow (���flow is the mental state in which players are so involved in something that nothing else mat- ters���).10 Quantitative measures for challenge and curiosity are inspired by previous work on entertainment metrics and extracted from the real-time player-opponent interaction.6 A mapping between the aforementioned factors and human notion of entertainment is derived using predator/prey games as an initial test-bed. Two neural network (NN) types, namely a feedforward NN and a fuzzy-NN, are trained through artificial evolution on gameplay experimental data to approximate the function be- tween the examined entertainment factors and player satisfaction. A comparison between the two methods is presented and the methods are validated against and compared with existing metrics of entertainment in the literature.11 Results demonstrate that both NNs map a function whose qualitative features are consistent with Malone���s corresponding en- tertainment factors and that the evolved feedforward NN provides a more accurate model of player satisfaction for predator/prey games than previous models designed for this genre of games.6 The generality of the proposed methodology and results that project its extensibility to other genres of digital entertainment are introduced here. More specifically, the quali- tative features of NN mappings between challenge, curiosity and entertainment presented in previous studies12 and here appear to generalize to games of the mixed-reality Playware interactive game platform.13 The paper concludes with a discussion of several remaining open questions regarding entertainment modeling and proposes future directions to answer these questions. The limitations of the presented methodology, and the extensibility of the proposed approach of entertainment capture and augmentation are also discussed.

March 14, 2007 17:14 WSPC/INSTRUCTION FILE SETN��Journal��v2 Modeling and Augmenting Game Entertainment through Challenge and Curiosity 3 2. Entertainment Modeling Research in the field of game AI is mainly focused on generating human-like (believ- able) and intelligent (see Ref. 1, 14 among others) characters. Complex NPC behaviors can emerge through various AI techniques however, there is no further analysis of whether these behaviors have a positive impact to the satisfaction of the player during play. Accord- ing to Taatgen et al. 15, believability of computer game opponents, which are generated through cognitive models, is strongly correlated with enjoyable games. Such implicit re- search hypotheses may well be true however, there is little evidence that specific NPCs generate enjoyable games unless a notion of interest or enjoyment is explicitly defined. There have been several psychological studies to identify what is ���fun��� in a game and what engages people playing computer games. Theoretical approaches include Malone���s principles of intrinsic qualitative factors for engaging gameplay,9 namely challenge, cu- riosity and fantasy as well as the well-known concepts of the theory of flow 10 incorporated in computer games as a model for evaluating player enjoyment, namely GameFlow.16 A comprehensive review of the literature on qualitative approaches for modeling player en- joyment demonstrates a tendency of overlapping with Malone���s and Csikszentmihalyi���s foundational concepts. These approaches include Lazzaro���s ���fun��� clustering based on four entertainment factors derived from facial expressions and data obtained from game surveys on players 17. According to Lazzaro, the four components of entertainment are: hard fun (related to the challenge factor of Malone), easy fun (related to the curiosity factor of Mal- one), altered states (i.e. ���the way in which perception, behavior, and thought combine in a collective context to produce emotions and other internal sensations��� ��� closely related to Malone���s fantasy factor) ��� and socialization (the people factor). Koster���s7 theory of fun, which is primarily inspired by Lazzaro���s four factors, defines ���fun��� as the act of mastering the game mentally. An alternative approach to fun measure is presented by Read et al.18 where fun is composed of three dimensions: endurability, engagement and expectations. Questionnaire tools and methodologies are proposed in order to empirically capture the level of fun for evaluating the usability of novel interfaces with children. Previous work in the field of quantitative entertainment modeling is based on the hy- pothesis that the player-opponent interaction ��� rather than the audiovisual features, the context or the genre of the game ��� is the property that primarily contributes the majority of the quality features of entertainment in a computer game. 6 Based on this fundamen- tal assumption, a metric for measuring the real-time entertainment value of predator/prey games was established as an efficient and reliable entertainment (���interest���) metric by val- idation against human judgement.19 According to this approach, the three qualitative cri- teria that collectively define entertainment for any predator/prey game are: the appropriate level of challenge, the opponent behavior diversity and the opponents��� spatial diversity. The quantifications of the three criteria provide an estimate ��� called the I (interest) value, that lies in [0,1] ��� of real-time entertainment which correlates highly with the human notion of entertainment. Moreover, a real-time learning mechanism based on neuron-evolution is used for augmenting the proposed I value. 19 Similar work in adjusting a game���s difficulty includes endeavors through reinforcement learning 20, genetic algorithms 21, probabilis-