Modeling emotions in simulated computer-mediated human-human interactions in a virtual game environment

Abstract

Emotions form a major part of humans’ day-to-day lives, especially in the areas of communication and interaction with others. They modify our gesture or facial expression and therefore serve as an additional communication channel. Furthermore, they have an impact on decision-making. This has two possible implications for computer science in the field of human-computer-interaction. First, computers should be able to adequately recognize and model human emotions if they genuinely want to help users in applied fields of human-human interactions. Second, a reliable and valid computer model of users’ emotions is the basis of effective implementations for human-computer interaction, with the computer thus being able to adapt to users’ emotions flexibly in any given application. From an empirical point of view, though, computerized recognition of human emotions still lacks substantial reliability and validity. In our opinion there are two main reasons for this shortcoming. First, triggers of emotional responses, i.e. eliciting situations, are typically complex in nature and thus difficult to predict or even assess once apparent. Second, the emotional response itself is a complex reaction involving subjects’ individual learning history, appraisal, preparedness, bodily reactions, and so forth. Both factors make it difficult for any algorithm to recognize real-life emotions. In a venture to approach this problem, the main goal of our study is to test an implementation of a computer model (COMPLEX) that predicts user emotions in a simulated human-human interaction. The prediction is supported by an elaborate appraisal model of emotions and the assessment of user bodily reactions, facial expression and speech. This article will give an overview of the theoretical background, the practical implementation of our new approach and first results of an empirical validation.