Natural gesture based interaction with virtual heart in augmented reality

Abstract

Augmented reality AR is a relatively new technology that blends digital content such as information, sound, video, graphics, or GPS data into our real world. The natural interaction of users with virtual content is one of the big challenges of the AR application. In this paper, we present a new approach to natural interaction with virtual objects employing users’ hands and fingers. We use a real-time image processing system to track gestures from users as a first step and to convert the gestures’ shapes on object commands as a second step. The paper describes how these techniques were applied in an interactive AR heart visualization interface. Our aim is to provide an interactive learning tool that will help students to learn about the components of the heart. Experimental results showed the effectiveness of the proposed method.