Hands-On: Using Gestures to Control Descriptions of a Virtual Environment for People with Visual Impairments

Abstract

Virtual reality (VR) uses three main senses to relay information: sight, sound, and touch. People with visual impairments (PVI) rely primarily on auditory and haptic feedback to receive information in VR. While researchers have explored several approaches to make navigation and perception of objects more accessible in VR, none of them offer a natural way to request descriptions of objects, nor control of the flow of auditory information. In this demonstration, we present a haptic glove that PVI can use to request object descriptions in VR with their hands through familiar hand gestures. We contribute designs for a set of hand gestures that allow PVI to interactively get descriptions of the VR environment. We plan to conduct an user study where we will have PVI interact with a VR environment using these gestures to request audio descriptions.