Developing intuitive gestures for spatial interaction with large public displays

Abstract

Freehand gestures used in gestural-based interactive systems are often designed around technical limitations of gesture capturing technologies, resulting in gestures that may not be intuitive to users. In this paper, we investigated freehand gestures that are intuitive to users with common technical knowledge. We conducted a gesture solicitation study with 30 participants, who were asked to complete 21 tasks on a large display using freehand gestures. All gestures in the study were video-recorded. We conducted in-depth interviews with each participant to ask about the gestures they had chosen and why they had chosen them. We found that a large proportion of intuitive freehand gestures had metaphoric origins from daily uses of two-dimensional surface displays, such as smart phones and tablets. However, participants may develop new gestures, particularly when objects they are manipulating deviated from those commonly seen in surface technologies. In this paper, we discuss when and why participants developed new gestures rather than reusing gestures of similar tasks on two-dimension surface displays. We suggest design implications for gestures for large public displays.