Effects of co-location and crossmodal interaction between Haptic, auditory and visual cues in presence

Abstract

The elicited sense of presence in a virtual environment (VE) is affected by the sensory cues provided during the interaction. Moreover multimodal integration may also be a contributing effect in this factor. The experiment presented analyzes the extent in which the addition of haptic, auditory and visual cues, as well as the integration that may take place between them, affects presence. We also analyze the effects of co-location between visual and haptic sensory modalities. Thus the experiment has a between subject design, where 16 subjects interact in a co-located condition (C)using the Reachin display and the other 16 in a non co-located condition (NC). The system used is a virtual version of the "Simon" game and subjects are requested to complete a memory task which consists in reproducing sequences, via selecting buttons. Results of this experiment have shown how firstly haptic cues are the principal modality for eliciting sense of presence and secondly the existence of differences in the benefits of multimodality between the two conditions. © 2008 Springer-Verlag Berlin Heidelberg.