Conversion of Sign Language to Text and Speech and Prediction of Gesture

Abstract

People with the inability to speak, use sign language for communication. Ordinary people usually find it very difficult to communicate with mute people due to their lack of understanding of the universal sign language. This paper aims to provide a solution for this very problem through a device that uses an Arduino Uno board, some flex sensors and an Android application to facilitate interaction amongst the users. The flex sensors detect the movements and gestures of the wearer and based on the established conditions for the different values generated, respective messages are sent using a Global System for Mobile (GSM) Module to the user’s android device which translates the text message to speech. The GSM module also attempts to create parameters for gesture predictions by sending sensor inputs to a cloud-based server for future reference. The application is ever learning and continues to evolve to be more reliable by examining user behaviours at all times. The use of this device allows mute people to convert sign language to speech, thereby making it significantly easier to talk to others, especially those who do not know sign language. This device empowers mute people and opens them up to previously unattainable opportunities.