Human target detection and localization with radars using deep learning

Abstract

In this contribution, we present a novel radar pipeline based on deep learning for detection and localization of human targets in indoor environments. The detection of human targets can assist in energy savings in commercial buildings, public spaces, and smart homes by automatic control of lighting, heating, ventilation, and air conditioning (HVAC). Such smart sensing applications can facilitate monitoring, controlling, and thus saving energy. Conventionally, the detection of radar targets is performed either in the range-Doppler domain or in the range-angle domain. Based on the application and the radar sensor, the angle or Doppler is estimated subsequently to finally localize the human target in 2D space. When the detection is performed on the range-Doppler domain, the processing pipeline includes moving target indicators (MTI) to remove static targets on range-Doppler images (RDI), maximal ratio combining (MRC) to integrate data across antennas, followed by constant false alarm rate (CFAR)-based detectors and clustering algorithms to generate the processed RDI detections. In the other case, the pipeline replaces MRC with Capon or minimum variance distortionless response (MVDR) beamforming to transform the raw RDI from multiple receive channels into raw range-angle images (RAI), which is then followed by CFAR and clustering algorithm to generate the processed RAI detections. However, in the conventional pipeline, particularly in case of indoor human target detection, both domains suffer from ghost targets and multipath reflections from static objects such as walls, furniture, etc. Further, conventional parametric clustering algorithms lead to single target splits, and adjacent target merges in the target range-Doppler and range-angle detections. To overcome such issues, we propose a deep learning-based architecture based on the deep residual U-net model and deep complex U-net model to generate human target detections directly from the raw RDI. We demonstrate that the proposed deep residual U-net and complex U-net models are capable of generating accurate target detections in the range-Doppler and the range-angle domain, respectively. To train these networks, we record RDIs from a variety of indoor scenes with different configurations and multiple humans performing several regular activities. We devise a custom loss function and apply augmentation strategies to generalize this model during real-time inference of the model. We demonstrate that the proposed networks can efficiently learn to detect and localize human targets correctly under different indoor environments in scenarios where the conventional signal processing pipeline fails.