A Robust Static Sign Language Recognition System Based on Hand Key Points Estimation

Abstract

Sign language recognition is not only an essential tool between normal people and deaf, but a prospective technique in human-computer interaction (HCI). This paper proposes a robust method based on the RGB sensor and hand key points estimation. Compared with depth sensor and the wearable devices, RGB sensor has smaller size and simpler operation process. With the hand key points detection technique, the data can conquer the influence of unfavourable factors like complex background, occlusion, and different angles. During training step, 5 kinds of machine learning algorithms are used for the classification of 20 letters in alphabet, and the highest classification accuracy are realized by SVM and KNN algorithms, which are 95.54% and 97.3% respectively. Finally, a real time sign language recognition system with SVM training model is built and it’s recognition accuracy can reach 97%, which confirms that our method can effectively eliminate unfavourable factors.