Sign Language in Immersive VR: Design, Development, and Evaluation of a Testbed Prototype

Abstract

Immersive Virtual Reality (IVR) systems support several modalities such as body, finger, eye, and facial expressions tracking, thus they can support sign-language-based communication. The combined utilization of tracking technologies requires careful evaluation to ensure high-fidelity transference of body posture, gestures, and facial expressions in real-time. This paper presents the design, development and evaluation of an IVR system utilizing state-of-the-art tracking options. The system is evaluated by certified sign language teachers to detect usability issues and examine appropriate methodology for large-scale follow-up evaluation by users fluent in sign language.