An optimized scheme of mel frequency cepstral coefficient for multi-sensor sign language recognition

Abstract

This paper proposes an optimized scheme of Mel Frequency Cepstral Coefficient (MFCC) and Deep-learning for sign language recognition using gyroscopes, accelerometers and surface electromyography, explores the possibility of building a sign language translation system by using these algorithms. Meanwhile, MFCC was compared with the traditional feature extraction methods, Linear Predictive Coding (LPC), Linear Predictive Cepstral Coefficient (LPCC), wavelet transform etc., in order to verify the effectiveness and superiority of MFCC for extracting characteristics of the surface of muscle. With continuous acquisition of surface electromyogram signals, an optimized scheme of MFCC was proposed to extract feature points. Then we cluster and classify the gesture features to match with the sign language words. The sign language words which can be matched successfully adjusts the language model training to be more consistent with the communication habits. This method provides a feasible way to realize a sign language recognition system which translates signs performed by deaf people into text/sound. The experimental results show that the accuracy of the feature extraction gesture recognition based on MFCC is about 90%, which is at least 4% higher than that of LPC, LPCC and wavelet transform. For translation of a complete statement, Bilingual Evaluation Understudy (BLEU) scores increases 5% after the adoption of the language model adjustment.