Context-responsive ASL Recommendation for Parent-Child Interaction

Abstract

Parental language input in early childhood plays a critical role in lifelong neuro-cognitive and social development. Deaf and Hard of Hearing (DHH) children are often at risk of language deprivation due to hearing parents' limited knowledge of sign language-the natural language for DHH children at birth. To offer an immersive sign language environment for DHH children, we designed a novel computer-mediated communication technology named Table Top Interactive System (TIPS). It aims to provide context-responsive recommendation of American Sign Language (ASL) in real-time for hearing parents during face-to-face joint play with their DHH children. The system emphasizes supporting parent autonomy by adapting ASL recommendations using parent's speech during play, and minimizes obtrusion for face-to-face interaction through an Augmented Reality (AR) display. This paper describes the design and development of an initial working prototype of TIPS and preliminary results of the system's efficiency regarding system latency and accuracy for ASL recommendation and visualization. Next, we plan to conduct a user study to gather expert and parent feedback about the system design and ASL recommendation strategies for long-term and personalized usage.