Creating Non-Visual Non-Verbal Social Interactions in Virtual Reality

Abstract

Although virtual reality (VR) was originally conceived of as a multi-sensory experience, most developers of the technology have focused on its visual aspects to the detriment of other senses such as hearing. This paper presents design patterns to make virtual reality fully accessible to non-visual users, including totally blind users, especially with non-verbal social interactions. Non-visual VR has been present in the blindness audio game community since the early 2000s, but the conventions from those interfaces have never been described to a sighted audience, outside of a few limited sonification interface papers. This paper presents non-visual design patterns created by five of the top English-speaking audio game developers through a three round Delphi method, encompassing 29 non-verbal social interactions grouped into 12 categories in VR, including movement, emotes, and self-expression. This paper will be useful to developers of VR experiences who wish to represent non-verbal social information to their users through non-visual conventions. These methods have only been rigorously tested through the commercial market, and not through scientific approaches. These design patterns can serve as the foundation for future investigation in exploring non-visual non-verbal social interactions in VR.