ASL-pro: American sign language animation with prosodic elements

Abstract

Efforts to animate American Sign Language (ASL) are aimed at eliminating barriers preventing ASL-using Deaf children from achieving their full potential. These barriers result from reduced information accessibility and affect language, cognitive development, and general education. Deaf children receive limited input with which they are unable to interact. Digital learning materials with ASL translation in the form of realistic 3D character animations can provide a solution to this problem. However, existing ASL animations lack the characteristics of natural, intelligible signing, producing stilted, robot-like signing when signs are put into sentences. What they lack is linguistic “prosody”: rhythmic phrasing, stress, and intonation. In ASL these markers are provided by timing and the face, head, and body. Our goal is to remedy this situation by adding predictable prosodic markers using algorithms that reflect the content and syntax of the signs in sequence. We refer to this new animation capability as ASL-pro, that is, ASL with pro(sody). Specifically, we describe (1) development of computational algorithms for inserting basic prosodic markers (e.g. facial articulations) and editing basic prosodic modifiers (e.g. lengthening sign duration in phrase-final position), and (2) design, development and initial evaluation of a new user interface allowing users to input English sentences in ASL-pro notation to automatically generate corresponding signing animations with prosodic elements.