Multisensory Experience for People with Hearing Loss: A Preliminary Study Using Haptic Interfaces to Sense Music

Abstract

In this paper, we describe a preliminary study on a multisensory music experience for people with hearing loss. Our main goal is to provide a music event through visual and tactile stimuli, granting a multisensory experience using haptic interfaces and taking advantage of visual feedback, vibrations and pressure to induce feelings. In this context, a mobile application was developed, allowing the user to interact with recorded audio samples that exploit vibrations to trigger emotions, such as fear, adrenaline, anxiety, suspense, drama, adventure, or even more complex moods like when dancing and relaxing. We thus describe our methodology (design, implementation and user assessment) for a preliminary study of a music experience based on a user-centered design approach. Indeed, we gathered promising results as the experience was considered effective and satisfying. We also uncovered some development issues to be addressed in future work, having to do with the use of specific hardware for providing a fully immersive experience.