Judgement as to whether or not people are "involved, enjoying and excited," based on the visual and physiological information

Abstract

We feel uncomfortable when we communicate with Embodied Conversational Agents (ECAs). This is because they are unable to express their mental state on Emotions using their method of communication adequately. Especially in multi-user interaction, ECAs have to detect the mental states of each user in order to interact naturally. In addition, the mental state of a person may be affected by extrinsic factors, such as the states of people around. Physiological indices are useful to understand the mental states of a person. It was impractical for ECAs to measure physiological indices of users. In this study, we experimentally investigated whether we could detect intrinsic "involvement, enjoyment, and/or excitement" ("I-measure") of a person and the extrinsic "I-measure." That was a social atmosphere of "I-measure," using visual information. As a result, both intrinsic and extrinsic "I-measure" could be detected by moving distances and speeds of a user's motion. Each threshold to detect motions was different: The threshold to detect extrinsic "I-measure" was lower than that of intrinsic "I-measure." We then proposed a method to detect the "I-measure" of a person and an atmosphere of "I-measure" in a multi-user interaction. © Springer-Verlag Berlin Heidelberg 2010.