Fingerspelling recognition using histogram of oriented point cloud vectors from depth data

Abstract

The high degree of freedom of hand movements produces a high variability of shapes and hand appearances that still challenges hand gesture recognition algorithms. This paper presents an approach to recognize sign language fingerspelling. Our approach, named histogram of oriented point cloud vectors (HOPC), is based on a new descriptor computed only from depth images. The segmented depth image is mapped into a 3D point cloud and divided into subspaces. In each subspace, 3D point vectors are mapped into their spherical coordinates around its centroid. Next, it is computed their orientations angles Hφ and Hθ onto two cumulative histograms. Normalized histograms are concatenated to form the image descriptor and used to train a Support Vector Machine classifier (SVM). To assess the feasibility of our approach, we evaluated it on a public data-set of American Sign Language (ASL) composed of more than 60,000 images. Our experiments showed a recognition accuracy average of 99.46%, achieving the state of the art.