Application of machine learning techniques for real-time sign language detection using wearable sensors

Abstract

Sign language is a method of communication primarily used by the hearing impaired and mute persons. In this method, letters and words are expressed by hand gestures. In fingerspelling, meaningful words are constructed by signaling multiple letters in a sequence. In this paper, a system has been developed to detect fingerspelling in American Sign Language (ASL) and Bengali Sign Language (BdSL) using (data) gloves containing some suitably positioned sensors. The methodologies employed can be used even in resource-constrained environments. The system is capable of accurately detecting both static and dynamic symbols in the alphabets. The system shows a promising accuracy of (up to) 96%. Furthermore, this work presents a novel approach to perform a continuous assessment of symbols from a stream of run-time data.