AudioTouch: Minimally invasive sensing of micro-gestures via active bio-acoustic sensing

Abstract

We present AudioTouch, a minimally invasive approach for sensing micro-gestures using active bio-acoustic sensing. It only requires attaching two piezo-electric elements, acting as a surface mounted speaker and microphone, on the back of the hand. It does not require any instrumentation on the palm or fngers; therefore, it does not encumber interactions with physical objects. The signal is rich enough to detect small diferences in micro-gestures with standard machine-learning classifers. This approach also allows for the discrimination of diferent levels of touch-force, further expanding the interaction vocabulary. We conducted four experiments to evaluate the performances of AudioTouch: a user study for measuring the gesture recognition accuracy, a follow-up study investigating the ability to discriminate diferent levels of touch-force, an experiment assessing the cross-session robustness, and, a systematic evaluation assessing the efect of sensor placement on the back of the hand.