Intuitive virtual objects manipulation in augmented reality: Interaction between user's hand and virtual objects

Abstract

Augmented Reality (AR) is a technological advancement which brings to reality the intuitive interaction that a user is attracted to virtual objects. In this paper, we focus on intuitive manipulation of virtual objects with bare hands. Specifically, we propose a method to deform, move, and join virtual objects by intuitive gestures, and aim at intuitive interactions with virtual objects. In order to manipulate virtual objects with bare hands, the position coordinates of the user's fingers are acquired using the Leap Motion Controller. Additionally, we use Bullet Physics as an engine to represent virtual objects. On the other hand, in AR technology, since the three-dimensional (3D) model is superimposed on the image of the real space afterward, it is always displayed on the front end rather than the user's hand (occlusion problem). Thus, the user cannot perform an intuitive manipulation. We solve the occlusion problem by performing appropriate hidden surface removal. From the results of a study using data gathered from a questionnaire survey, we found support for the intuitive interaction between the user and virtual objects.