Real-time dynamic gesture recognition and hand servo tracking using PTZ camera

Abstract

A technology of real-time dynamic gesture recognition and hand tracking using a Pan-Tilt-Zoom (PTZ) camera was presented in this study. It was aimed to achieve robust scheme that stably recognized simple hand gestures and tracked the hand by means of a PTZ camera to keep the fingertip remaining in the center of the camera. For this purpose, the hand region was initially segmented in a cluttered environment using skin color segmentation in YCbCr color space to get the silhouette of the hand. Furthermore, the Monte Carlo Sampling method was used to estimate the Cubic Bezier curves best fitted to the sub contour points centralized in each contour point, and the fingertips were detected by combining the local maximums of a cumulative curvature with detection of convex defects. After that, feature triangle analysis was utilized to achieve dynamic recognition of simple gestures including “right click down” and “right click up”. Finally, the PTZ camera was driven by the algorithm to achieve servo tracking with the target fingertip when the gesture “right click down” was detected. As is shown by the experimental results, the proposed approach recognized the dynamic gestures and located the fingertips’ positions precisely, and realized the follow-up servo tracking using PTZ camera in real-time.