SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational Settings

Abstract

Deaf and hard-of-hearing (DHH) students often encounter challenges in their specialized educational settings, particularly in language literacy development, due to limited exposure to written and spoken language. We, therefore, collaborated with 8 DHH students and 2 Teachers of the Deaf (ToDs) from a School for Deaf in Sri Lanka to explore how to effectively utilize a real-time augmented reality (AR) caption interface in their classroom settings, aiming to enhance their learning experience. We adopted a User- Centered Design (UCD) process to develop and refine the prototype addressing the design challenges affecting the overall learning experience. User study with 12 DHH participants revealed a strong preference (91.66%) for our prototype and highlighted its potential to enhance the learning experiences. Our findings contribute to further exploration of the potential of tailored AR caption interfaces in addressing the educational challenges faced by DHH students and enhancing their learning experience in specialized educational settings.