SignFind: A Synchronized Sign Language and Chinese Character Teaching Game for Chinese Deaf Children Using Gesture Recognition

Abstract

This paper presents a targeted system to help Chinese deaf children learn both sign language and Chinese characters in early language learning. The system combines sign recognition and in-air writing techniques with games so that children can practice sign language and Chinese character writing at the same time. Sign recognition is used to assess the accuracy of sign language and in-air writing records the process of writing Chinese characters. In addition, the game adds to the learning fun and makes children more willing to complete learning tasks. We have developed a prototype to evaluate the effectiveness of a simultaneous Chinese sign language and Chinese character teaching system based on gesture recognition and in-air writing. We expect that this system will increase children's willingness and efficiency to learn sign language and Chinese characters, and eventually be used to assist in early language education for deaf children in the Chinese region.