Avatar Interpreter: Improving Classroom Experiences for Deaf and Hard-of-Hearing People Based on Augmented Reality

Abstract

Deaf and hard-of-hearing (DHH) people experience difficulties in accessing education. One reason is that they miss out on oral information in the classroom. Although some classrooms are equipped with sign language interpreters specifically for deaf students, DHH students have trouble switching their gaze and attention when they cannot see the interpreter and the visual information in the classroom simultaneously. To address this challenge, this paper develops Avatar Interpreter, a visual interface for synchronized sign language interpreters on an augmented reality head-mounted display. Avatar Interpreter can be displayed at different sizes and in different locations in space to fit different scenarios, helping deaf students to better receive sign language information and enhancing the classroom experience for deaf students. In the work presented in this paper, we discuss our prototype, make design recommendations, and discuss configuration and implementation. Finally, we propose questions and research methods for an upcoming user study.