Unsupervised learning based static hand gesture recognition from RGB-D sensor

Abstract

In this paper, we propose a novel, real-time static hand gesture recognition framework based on unsupervised learning. We use unlabelled training data from an RGB-D (RGB, Depth) sensor and use the depth data to detect the hand(s) in the image. We believe that the gesture is dependent on the shape of the hand, which is defined by number of extended/open fingers and which fingers are extended/open. We use unsupervised learning techniques to detect and label the extended fingers in the training data. We also use the same algorithm to detect and label the extended fingers in the test images on-the-fly. Our gesture recognition system is independent of the orientation of the gesturing hand and arm. Moreover, it does not require any markers and is completely unobtrusive, making it suitable for assistive living paradigms.