Arabic sign language recognition using optical flow-based features and HMM

Abstract

Sign language is the main communication channel of deaf community. It uses gestures and body language such as facial expressions, lib patterns, and hand shapes to convey meaning. Sign language differs from one country to another. Sign language recognition helps in removing barriers between people who understand only spoken language and those who understand sign language. In this work, we propose an algorithm for segmenting videos of signs into sequences of still images and four techniques for Arabic sign language recognition, namely Modified Fourier Transform (MFT), Local Binary Pattern (LBP), Histogram of Oriented Gradients (HOG), and combination of HOG and Histogram of Optical Flow (HOG-HOF). These techniques are evaluated using Hidden Markov Model (HMM). The best performance is obtained with MFT features with 99.11% accuracy. This recognition rate shows the correctness and robustness of the proposed signs’ video segmentation algorithm.