FMCW Radar-Based Real-Time Hand Gesture Recognition System Capable of Out-of-Distribution Detection

Abstract

Real-time hand gesture recognition plays a vital role in human-computer interaction (HCI). Recent radar-based hand gesture recognition methods have focused on achieving high classification accuracy using deep neural network (DNN)-based classifiers. However, the hand gesture recognition system should not only classify the gestures accurately but also detect out-of-distribution (OOD) samples to be used in real-world HCI scenarios with high reliability. Recognition systems without OOD detection capability misclassify unintended gestures in silence, especially in real-time scenarios. To tackle this problem, we propose a real-time hand gesture recognition system that can simultaneously classify hand gestures and detect OOD samples by using a Frequency Modulated Continuous Wave (FMCW) radar sensor. First, we design radar data processing technique and Transformer encoder-based classifier to achieve high classification accuracy. Second, the relative Mahalanobis distance (RMD)-based OOD detection method is adopted to increase the reliability of the proposed system. Finally, one in-distribution dataset and two OOD datasets are collected to verify the proposed system. The proposed system achieves a classification accuracy of 93.95% on the in-distribution dataset. We conduct the OOD detection experiments with two OOD datasets for which the proposed system reports AUROC values of 92.96% and 92.84%, respectively. Furthermore, the feasibility of the proposed system is certified through a real-time experimental demonstration.