Can haptic feedback improve gesture recognition in 3D handwriting systems

Abstract

Current gesture interfaces accept relatively simple postures and motions for reliable inputs, which are still far from natural and intuitive experiences for the users. This paper suggests a unique idea that haptic feedback has a potential to improve not only user experiences but also gesture recognition performances. We expect haptic feedback to provide users cues for natural writing movement and accordingly generate movement easily recognized by the system. We developed a writing gesture recognition system using the K-Means clustering algorithm for writing state estimation and a haptic feedback system which involved frictional sensation during writing and impulsive sensation at the beginning and ending of writing. The experiments on five participants showed an approximately 5% and 4% improvement in the true positive and the false negative gesture recognition rate with visual-haptic feedback compared to visual feedback alone. We confirmed that the improvement was due to changes in hand motion by haptic feedback, which led to a higher correlation between reference waveform and performed motion and an increase of the finger stopping time at the end of the writing. We also confirmed the positive effects of our haptic feedback on the user experiences.