A real-time hand detection system during hand over face occlusion

Abstract

This paper devises a vision-based hand detection system which can handle the situation of the hand and face occlusion. Firstly , human face is coarsely located by skin color detection and ellipse template matching and further determined by using the grayscales and positions of human eyes. Secondly, when the hand occludes the partial region of the face, a novel hand detection method based on Camshift tracking algorithm, force field method and Sobel edge extraction is developed to segment the hand. Finally, the positions of segmented hands are sent to the computer for controlling the cursor movement. In order to reduce the cursor jitter caused by wrong hand detection and generate a smooth trajectory of the cursor , the coordinates of hand positions are corrected by combining the least squares fitting method of orthogonal polynomial and an adaptive Catmull-Rom interpolation algorithm. Experimental results showed that the proposed method could detect hands accurately with the run time of 0.08s per frame and demonstrated a significant improvement in performance when the hand enters the face region. Moreover, our system accomplished smooth movements of the cursor by the vision-based hand detection.