Advancing robust human activity recognition via informative mmWave radar characteristics and a lightweight spatio-spectro-temporal network

Abstract

Human activity recognition (HAR) is increasingly important in aiding our daily life, with millimeter-wave (mmWave) radar sensors emerging as a promising noninvasive solution thanks to their excellent spatial and velocity resolution. Although existing radar-based systems have shown strong performance, they primarily focus on micro-Doppler signatures while neglecting angle information, which can hinder practical deployment in real-world scenarios. Moreover, current state-of-the-art recognition models using mmWave radar often require substantial computational resources, making integration into resource-constrained devices challenging. This work proposes an efficient radar-based HAR system that leverages angle and spectro-temporal information from micro-Doppler signatures. Our system utilizes a multi-channel micro-Doppler representation corresponding to the number of virtual antenna receivers as input. Then, a lightweight dilated convolutional network, namely SST-DCN, extracts spatial-aware multi-scale spectro-temporal information through time-frequency dilated convolutions. Experimental results on our real-world dataset demonstrate the superiority of our approach compared to conventional features and other state-of-the-art radar-based HAR systems.