Gesture recognition and machine learning applied to sign language translation

Abstract

In this paper we propose an intelligent system for translating sign language into text. This approach consists of hardware and software. The hardware is formed by flex, contact, and inertial sensors mounted on a polyester-nylon glove. The software consists of a classification algorithm based on the k-nearest neighbors, decision trees, and the dynamic time warping algorithms. The proposed system is able to recognize static and dynamic gestures. This system can learn to classify the specific gesture patterns of any person. The proposed system was tested at translating 61 letters, numbers, and words from the Ecuadorian sign language. Experimental results demonstrate that our system has a classification accuracy of 91.55%. This result is a significant improvement compared with the results obtained in previous related works.