How low level observations can help to reveal the user's state in HCI

Abstract

For next generation human computer interaction (HCI), it is crucial to assess the affective state of a user. However, this respective user state is - even for human annotators - only indirectly inferable using background information and the observation of the interaction's progression as well as the social signals produced by the interlocutors. In this paper, coincidences of directly observable patterns and different user states are examined in order to relate the former to the latter. This evaluation motivates a hierarchical label system, where labels of latent user states are supported by low level observations. The dynamic patterns of occurrences of various social signals may in an integration step infer the latent user's state. Thus, we expect to advance the understanding of the recognition of affective user states as compositions of lower level observations for automatic classifiers in HCI. © 2011 Springer-Verlag.