Arabic sign language characters recognition based on a deep learning approach and a simple linear classifier

Abstract

One of the best ways of communication between deaf people and hearing people is based on sign language or so-called hand gestures. In the Arab society, only deaf people and specialists could deal with Arabic sign language, which makes the deaf community narrow and thus communicating with normal people difficult. In addition to that, studying the problem of Arabic sign language recognition (ArSLR) has been paid attention recently, which emphasizes the necessity of investigating other approaches for such a problem. This paper proposes a novel ArSLR scheme based on an unsupervised deep learning algorithm, a deep belief network (DBN) coupled with a direct use of tiny images, which has been used to recognize and classify Arabic alphabetical letters. The use of deep learning contributed to extracting the most important features that are sparsely represented and played an important role in simplifying the overall recognition task. In total, around 6,000 samples of the 28 Arabic alphabetic signs have been used after resizing and normalization for feature extraction. The classification process was investigated using a softmax regression and achieved an overall accuracy of 83.32%, showing high reliability of the DBN-based Arabic alphabetical character recognition model. This model also achieved a sensitivity and a specificity of 70.5% and 96.2%, respectively.