Sky Writer: Towards an Intelligent Smart-phone Gesture Tracing and Recognition Framework

Abstract

We present Sky Writer, an intelligent smartphone gesture tracking and recognition framework for free-form gestures. The design leverages anthropomorphic kinematics and device orientation to estimate the trajectory of complex gestures instead of employing traditional acceleration based techniques. Orientation data are transformed, using the kinematic model, to a 3D positional data stream, which is flattened, scaled down, and curve fitted to produce a gesture trace and a set of accompanying features for a support vector machine (SVM) classifier. SVM is the main classifier we adopted but for the sake of comparison, we couple our results with the hidden Markov models (HMM). In this experiment, a dataset of size 1200 is collected from 15 participants that performed 5 instances for each of 16 distinct custom developed gestures after being instructed on how to handle the device. User-dependent, user-independent, and hybrid/mixed learning scenarios are used to evaluate the proposed design. This custom developed gesture set achieved using SVM 96.55%, 96.1%, and 97.75% average recognition rates across all users for the respective learning scenarios.