Multimodal interaction concepts for mobile augmented reality applications

Abstract

Augmented reality on mobile phones - i.e. applications where users look at the live image of the device's video camera and the scene that they see is enriched by 3D virtual objects - provides great potential in areas such as cultural heritage, entertainment, and tourism. However, current interaction concepts are often limited to pure 2D pointing and clicking on the device's screen. This paper explores different interaction approaches that rely on multimodal sensor input and aim at providing a richer, more complex, and engaging interaction experience. We present a user study that investigates the usefulness of our approaches, verify their usability, and identify limitations as well as possibilities for interaction development for mobile augmented reality. © 2011 Springer-Verlag Berlin Heidelberg.