Multi-Hand Gesture Recognition Using Automotive FMCW Radar Sensor

Abstract

With the development of human–computer interaction(s) (HCI), hand gestures are playing increasingly important roles in our daily lives. With hand gesture recognition (HGR), users can play virtual games together, control the smart equipment, etc. As a result, this paper presents a multi-hand gesture recognition system using automotive frequency modulated continuous wave (FMCW) radar. Specifically, we first constructed the range-Doppler map (RDM) and range-angle map (RAM), and then suppressed the spectral leakage, and dynamic and static interferences. Since the received echo signals with multi-hand gestures are mixed together, we propose a spatiotemporal path selection algorithm to separate the mixed multi-hand gestures. A dual 3D convolutional neural network-based feature fusion network is proposed for feature extraction and classification. We developed the FMCW radar-based platform to evaluate the performance of the proposed multi-hand gesture recognition method; the experimental results show that the proposed method can achieve an average recognition accuracy of 93.12% when eight gestures with two hands are performed simultaneously.