Hand gesture recognition based on saliency and foveation features using convolutional neural network

Abstract

The rapid growth in multimedia technology opened new possibilities for effective human–computer interaction (HCI) using multimedia devices. Gesture recognizing is an important task in the development of an effective human–computer interaction (HCI) system and this work is an attempt in this direction. This work presents an approach for recognizing hand gestures using deep convolutional neural network architecture from features derived from saliency and foveation information. In contrast to the classical approach of using raw video stream as input, low-level features driven by saliency and foveation information are extracted from depth video frames are considered for gesture recognition. The temporal variation of these features is given as input to a convolutional neural network for recognition. Due to the discriminative feature learning capability of deep learning models and the invariance of depth information to visual appearance, high recognition accuracy was achieved by this model. The proposed approach achieved an accuracy of 98.27% on SKIG hand gesture recognition dataset.