Investigation of feature elements and performance improvement for sign language recognition by hidden Markov model

Abstract

Sign language is commonly used as one means of communication for hearing-impaired or speech-impaired people. However, there are many difficulties in learning sign language. If automatic translation for sign language can be realized, it would be extremely valuable and helpful not just to those who are physically impaired but to unimpaired people as well. The cause of the difficulty in automatic translation is that there are many kinds of specific hand motions and shapes, which make it difficult to discriminate each motion. Consequently, this has a negative impact on accurate recognition. This paper presents a recognition method that is able to maintain accurate recognition of different signs that encompass a multitude hand motions and shapes. The main feature of our approach is the use of colored gloves to detect hand motions and shapes. For our investigation, a recognition scheme using HMM (Hidden Markov Model) has been introduced to enhance recognition performance. In this scheme, performance depends on the feature elements extracted from each sign language motion. Feature elements of sign language motions and their unification are investigated, and the recognition performance is clarified using these feature elements and compared with each result. Although the percentage of recognition successes for each feature element is low, from 21.7% to 42.7%, it was shown that recognition success for the combined element results increased from 55.2% to 61.9% for 25 different sign language motions. In addition, the removal of candidates was also examined to enhance performance as a form of preprocessing using a threshold obtained from DP matching. It is also confirmed through experiments that the recognition success rate increased by a few percentage.