Human gesture recognition using 3.5-dimensional trajectory features for hands-free user interface

  • Takahashi M
  • Fujii M
  • Naemura M
 et al. 
  • 15


    Mendeley users who have this article in their library.
  • 8


    Citations of this article.


We present a new human motion recognition technique for a hands-free user interface. Although many motion recognition technologies for video sequences have been reported, no man-machine interface that recognizes enough variety of motions has been developed. The difficulty was the lack of spatial information that could be acquired from video sequences captured by a normal camera. The proposed system uses a depth image in addition to a normal grayscale image from a time-of-flight camera that measures the depth to objects, so various motions are accurately recognized. The main functions of this system are gesture recognition and posture measurement. The former is performed using the bag-of-words approach. The trajectories of tracked key points around the human body are used as features in this approach. The main technical contribution of the proposed method is the use of 3.5D spatiotemporal trajectory features, which contain horizontal, vertical, time, and depth information. The latter is obtained through face detection and object tracking technology. The proposed user interface is useful and natural because it does not require any contact-type devices, such as a motion sensor controller. The effectiveness of the proposed 3.5D spatiotemporal features was confirmed through a comparative experiment with conventional 3.0D spatiotemporal features. The generality of the system was proven by an experiment with multiple people. The usefulness of the system as a pointing device was also proven by a practical simulation.

Author-supplied keywords

  • all or part of
  • bag-of-words
  • depth information
  • gesture recognition
  • or hard copies of
  • permission to make digital
  • this work for

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Get full text


  • Masaki Takahashi

  • Mahito Fujii

  • Masahide Naemura

  • Shin'ichi Satoh

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free