Gesture recognition method utilizing ultrasonic active acoustic sensing

Abstract

We propose a method for gesture recognition that utilizes active acoustic sensing, which transmits acoustic signals to a target, and recognizes the target’s state by analyzing the response. In this study, the user wore a contact speaker that transmitted ultrasonic sweep signals to the user’s body and a contact microphone that detected the ultrasound propagated through the body. The propagation characteristics of the ultrasound changed depending on the user’s movements. We utilized these changes to recognize the user’s gestures. One of the important novelty features of our method is that the user’s gestures can be acquired not only from the physical movement but also from the user’s internal state, such as muscle activity, since ultrasound is transmitted via both the user’s internal body and body surface. Moreover, our method is not adversely affected by audible-range sounds generated by the environment and body movements because we utilize ultrasound. We implemented a device that uses active acoustic sensing to effectively transmit/detect the ultrasound to/from the body and investigated the performance of the proposed method in 21 contexts with 10 subjects. The evaluation results confirmed that the precision and recall are 93.1% and 91.6%, respectively when we set 10% of the data as training data and the rest as testing data in the same data set. When we used the data set for training and the other data set for testing in the same day, the precision and recall are 51.6% and 51.3%, respectively.