Hand Gesture to Speech and Text Conversion Device

Abstract

Sign language and facial expressions are the major means of communication for the speech flawed people. General people can understand the facial expression to an extent but cannot understand the sign language. Dumb people are unable to express their thoughts to normal humans. To reduce this gap of communication, this paper presents an electronic system which will help the mute people to exchange their ideas with the normal person in emergency situations. The system consists of a glove that can be worn by the subject which will convert the hand ges-tures to speech and text. The message displayed will also help deaf people to understand their thoughts. This prototype involves raspberry pi 3 as the micro-controller along with the flex sensors, accelerometer sensor. The resistance of the flex sensor changes due to the bending moment of the fingers of the subject. The accelerometer measures the angular displacement of the wrist along the y-axis. The microcontroller takes the input from the two sensors and matches it with the pre-programmed values and plays the respective message. The system makes use of python and its libraries for microcontroller programming.