SoundVizVR: Sound Indicators for Accessible Sounds in Virtual Reality for Deaf or Hard-of-Hearing Users

Abstract

Sounds provide vital information such as spatial and interaction cues in virtual reality (VR) applications to convey more immersive experiences to VR users. However, it may be a challenge for deaf or hard-of-hearing (DHH) VR users to access the information given by sounds, which could limit their VR experience. To address this limitation, we present "SoundVizVR", which explores visualizing sound characteristics and sound types for several types of sounds in VR experience. SoundVizVR uses Sound-Characteristic Indicators to visualize loudness, duration, and location of sound sources in VR and Sound-Type Indicators to present more information about the type of the sound. First, we examined three types of Sound-Characteristic Indicators (On-Object Indicators, Full Mini-Maps and Partial Mini-Maps) and their combinations in a study with 11 DHH participants. We identified that the combination of Full Mini-Map technique and On-Object Indicator was the most preferred visualization and performed best at locating sound sources in VR. Next, we explored presenting more information about the sounds using text and icons as Sound-Type Indicators. A second study with 14 DHH participants found that all Sound-Type Indicator combinations were successful at locating sound sources.