Integrated deep learning structures for hand gesture recognition

Abstract

In this paper, object control with hand movements is proposed for controlling distant objects even when the user is far from the system. This method is based on finding hands, predicting the states and direction of the hand movement. This human-computer interface (HCI) is an assistive system for users at near/far from objects. The model is specifically designed for controlling computer mouse on big screens during the formal presentation. By moving the hand to the left, right, up and down moves the mouse pointer and sends mouse button command using hand states. Here, close hand triggers mouse button, until the same hand is opened. In this system, Single Shot Multi Box Detection (SSD) architecture is used for object detection and Convolutional Neural Network (CNN) is used for predicting hand states. This integrated system allows users to control the mouse from a distant position without using any hardware. The test results show that this system is robust and accurate. This invention uses a single camera and aids users who are far from the computer during a presentation to shuffle through the slides.