Hand gesture recognition for sign language: A skeleton approach

Abstract

In this work, we propose two novel approaches to classify sign images based on their skeletons. In the first approach, the distance features are extracted from the endpoints and junction points remarked on the skeleton. The extracted features are aggregated by the use of interval-valued type data and are saved in the knowledge base. A symbolic classifier has been used for the purpose of classification. In second approach, the concept of spatial topology is combined with the symbolic approach for classifying the sign gesture skeletons. From the end points and junction points of skeletons, triangles are generated using Delaunay Triangulation; and for each triangle, features like lengths of each side and angles are extracted. These extracted features of each skeleton of signs are clumped and represented in the form of interval-value type data. A suitable symbolic classifier is designed for the purpose of classification. Experiments are conducted on our own real dataset to evaluate the performance of two approaches. The experimental results disclose the success of the proposed classification approach.