Multi-angle fusion for low-cost near-field ultrasonic in-air gesture recognition

4Citations
Citations of this article
15Readers
Mendeley users who have this article in their library.

This article is free to access.

Abstract

With the growing interest in Human-Machine interface (HMI), an increasing amount of effort is made to provide always-on low-cost touchless control solutions for Internet-of-Things (IoT) edge devices. In this study, we explore near-audio ultrasound for in-air ultrasonic gesture design and recognition. We propose to use beamforming followed by stages of feature extraction and a Temporal Convolution Network (TCN) for classification. The study is applied to a small form factor concentric hexagonal array of 7 microphones where a beamforming stage is leveraged for spatial feature extraction and ultrasonic gesture features fusion from different angles. In case of such limited number of microphones, we show that a customized Filter-and-Sum (FaS) beamformer is well suited for this application with a set of 5-tap filters. We optimize the beamformer by fitting a fixed beam in the ultrasonic frequency domain, to make the ultrasonic band of interest (18 kHz - 24 kHz) available for use. The beamformer generates parallel readings in time of Doppler shifts from a set of assigned angles as a hand gesture is performed near the array. A TCN of only 10k parameters is used to classify these parallel readings into predefined symbols to build a gesture alphabet. The TCN operates on two modes sharing the same TCN structure with the option to switch between them by loading different set of coefficients. Features from concatenated beamformed frequency points are learned with a per symbol classification accuracy in the range 92%-100% computed on a test set and visualized in the form of normalized confusion matrix. The proposed system gives users a degree of fiexibility where gesture diversity can be obtained by grouping the trained symbols from the built alphabet in a post-training design stage. This paves the way for fiexible, intuitive and easy to remember gestures.

Cite

CITATION STYLE

APA

Ibrahim, E. A., Geilen, M., Li, M., & De Gyvez, J. P. (2020). Multi-angle fusion for low-cost near-field ultrasonic in-air gesture recognition. IEEE Access, 8, 191204–191218. https://doi.org/10.1109/ACCESS.2020.3031677

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free