The effective combination of haptic and auditory textural information

Abstract

With the increasing availability and quality of auditory and haptic means of interaction, it is not unusual to incorporate many modalities in interfaces rather than the purely visual. The user can be powerfully affected however when information presented in different modalities are combined to become multimodal. Providing interface designers with the means to implement haptic-audio interfaces might result in adverse effects to interaction unless they are also equipped with structured knowledge on how to select effective combinations of such information. This work introduces `Integration of Information´ as one important dimension of haptic-audio interaction and explores its effects in the context of multimodal texture perception. The range and resolution of available textures through force feedback interaction is a design consideration that might benefit from the addition of audio. This work looks at the effect of combining auditory and haptic textures on people's judgment of the roughness of a virtual surface. The combined haptic-audio percepts will vary in terms of how congruent they are in the information they convey regarding the frequency of bumps or ridges on the virtual surface. Three levels of integration (conflicting, redundant, or complementary) are described and their possible implications discussed in terms of enhancing texture perception with force-feedback devices.