Semiotic analysis for gestural and emotional human-computer interaction

Abstract

The discussion that follows describes the design of an interactive artwork To be or not to be using semiotics. The goal of To be or not to be was to create a user experience that was coherent and continuous, generating gestures and emotions - not uncommon Human Computer Interaction (HCI) objectives. The design problem was to isolate what could be elements of user experience as inputs and outputs in a multimedia interactive system. Essentially there were five parts to the process: the first was an understanding of gesture and its modeling within the framework of generating expressive gestures in theatre - defining media before simulation; the second, was to define the input and output process by which gestural interaction using HCI media might proceed; the third was to create a semiotic matrix of both the theatrical and HCI terms as equivalences, creating a system by which the design could follow; the fourth was evolving an experience, in this case an interactive film-game, that generated gestures and associated emotional content; the fifth was a user evaluation and statistical analysis (results summary only). The emphasis presented here is on the preparatory stage of correct process modeling, leading to the effective application of semiotic analysis. Readers are encouraged to access the URL for youtube description of the work. http://www.youtube.com/watch?v=jKNvSpXG0Z0 © 2013 Springer-Verlag Berlin Heidelberg.