From Exploration of Virtual Replica to Cultural Immersion Through Natural Gestures

Abstract

We investigate in this work the potential of multimodal rendering for assisting users during culturally-related navigation and manipulation tasks inside virtual environments. We argue that natural gestures play an important role for engaging users in experiencing the cultural dimension of a given environment. To this end, we propose an open system for multi-user visualization and interaction that enables users to employ natural gestures. We explored different configurations and controls in order to achieve the most accurate and natural user experience. One being switching between the navigation and manipulation mode based on distance and orientation towards different points of interest and the other being based on interacting with a virtual UI used for switching between the two modes. We also implemented both a single-user and a multi-user version. The single-user version having a normal, computer monitor based, point of view is better for a more accurate and detailed viewing experience. Also, in this version the user would be wearing the Myo armband and also using the Leap Motion for a more immersed experience. The multi-user version is based on a holographic pyramid which has two user perspectives, one of the Myo user and the other being the Leap Motion user’s, and two for the spectators’ point of view. Finally, we discuss findings on the users’ perceptions of experienced cultural immersion.