Real-time robot manipulation using mouth gestures in facial video sequences

Abstract

In this paper a novel method for the automatic command of three degrees of freedom of a robot using mouth gestures is presented. The method uses a set of different pixel-based segmentation algorithms and morphological restrictions in order to extract the mouth area from the frames. A fuzzy inference system is then used in order to produce a small subset of discriminante features for gesture classification. A state machine was designed in order to stabilize the robot command task by using a temporal sliding mean on the detected gestures. Experimental results show that the method is both robust and reliable when operated by different people, and fast enough to keep the detection's rate in real-time. © 2007 Springer-Verlag Berlin Heidelberg.