Sensory Properties in Fusion of Visual/Haptic Stimuli Using Mixed Reality

Abstract

This chapter introduces our experimental investigation about sensory properties in the fusion of visual/haptic stimuli by using Mixed-Reality (MR) technique. Especially, we focus on the discrepancy between the two stimuli. When making a purchase from TV or online, we are sometimes disappointed by a product whose actual scale and material differ from our image, even though its appearance originally impressed us. This case indicates the use of integrated multiple sensory cues that include not only the visual but also the auditory and haptic to extract the properties of objects. However, the fusion of multiple sensory cues by interaction with each other has not been well examined. We introduce our developed system that can independently control the sensibility parameters of visual and haptic cues to study the effect of these cues on sensory properties. MR techniques have been applied in factories to assist assembly, inspection and maintenance operations (Ohta & Tamura, 1999) (Wiedenmaier, et al, 2001) (Friedrich, 2002) (Fiorentino et al, 2002) (Nolle & Klinker, 2006). Recently, designing operations for industrial products are gathering attention as the next generation of MR applications (Navab, 2003) (Lee & Park, 2005) (Sandor et al, 2007). In ordinary designing operations, first, designers develop a broad plot of shape, appearance, and inner structure using a Computer Aided Design (CAD) system. After that, a mock-up, which is a dummy model of the designing product, is generated to examine such detailed information as a sense of touch and surface texture that is difficult to represent by CAD. However, generating a mock-up is very expensive, especially since it uses temporal resources. Thus, it is not practical to regenerate a mock-up whenever a design receives a minor change. The display system shown in Figure 1 might create a new style of product design to reduce operation processes. For example, in ordinary product design, when a designer wants to evaluate different impressions caused by subtle changes of surface material, many similar preproduction samples must be generated that correspond to each change. However, design variations are usually limited because they are too expensive. On the other hand, using our proposed system, evaluation is possible by superimposing computer graphics (CG) textures of various appearances onto a design mock-up that solves not only cost problems but also the limitations of trial design variations. 30