GestTalk—Real-Time Gesture to Speech Conversion Glove

Abstract

It is often observed that people who are speech or hearing impaired find it difficult to communicate with others. According to WHO [5], over 5% of the global population has disabling hearing loss, while many more are speech impaired. These people mainly rely on sign languages for their daily communication. Sign language is quite complicated for an average person to understand, which makes the world less accessible to a person who has acquired this disability. Hence, to solve this existing problem and make the world more accessible for such people, we propose GestTalk, a smart glove specifically designed to enable speech-impaired people to communicate with others by translating their performed gesture to speech with the help of machine learning and IoT. We are using hardware-based glove loaded with home-made flex sensor to capture data points and machine learning algorithm to map these gestures to speech dynamically in real time. Usage of home-made flex sensor, each costing under 10 INR, plays the vital role in capturing finger movement during the gesture, and certainly makes GestTalk a cost-effective gadget. We were able to achieve ~95% of prediction accuracy on the trained sample with our minimal prototype. We present this case study to show how GestTalk can aid the hearing/speech impaired by enabling them to communicate with the world via speech cheaply and economically.