Freehand gesture-based 3D manipulation methods for interaction with large displays

Abstract

Gesture-based 3D interaction is a research topic with application in numerous scenarios which gained relevance with the recent advances in low-cost tracking systems. Yet, it poses many challenges due to its novelty and consequent lack of systematic development methodologies. Developing easy to use and learn gesture-based 3D interfaces is particularly difficult since the most adequate and intuitive gestures are not always obvious and there is often a variety of different gestures used to perform similar actions. This paper presents the development and evaluation of interaction methods to manipulate 3D virtual objects in a large display set-up using freehand gestures detected by a Kinect depth sensor. We describe the implementation of these methods and the user studies conducted to improve them and assess their usability as manipulation methods. Based on the results of these studies we also propose a method that overcomes the lack of roll movement detection by the Kinect and makes simpler the scaling and rotation in all degrees-of-freedom using hand gestures.