Assessing the efficiency of using augmented reality for learning sign language

Abstract

In this study we examined whether the success rate regarding accuracies of signing particular words differs when the signs for the words are acquired either from (a) a picture symbolizing a sign, (b) an Augmented Reality mobile application, or (c) a physically present sign language interpreter. We analyzed whether any differences would appear between the 25 people included in an experiment. We used three pairs of words and the participants were accordingly classified into three groups. Each group was asked to sign one pair of words based on acquiring signs either from pictures, the Augmented Reality mobile application or a sign language interpreter. When the participants signed single words, their accuracies (=success rates) were evaluated by two sign language interpreters. The results revealed the lowest success rates when watching pictures, while the success rates improved by 35% when using the Augmented Reality mobile application. When a sign-language interpreter signed words the participants’ success rates in signing increased by an additional 9%. No differences were found between D/HH signers and hearing non-signers. Generally, participants were the least successful when signing the words “break” and “claw”.