A Progressive Target-Aware Network for Drone-Based Person Detection Using RGB-T Images

Abstract

Highlights: What are the main findings? A novel drone-based person detection model, PTANet, is proposed. It integrates a global adaptive feature fusion module (GAFFM), a person segmentation auxiliary branch (PSAB) during training, and a cross-modality background mask (CMBM) during inference to progressively focus on person targets while suppressing background clutter. The proposed PTANet consistently outperforms state-of-the-art cross-modality target detection methods on three person detection datasets and demonstrates strong potential for real-time on-board deployment. What is the implication of the main finding? The proposed PTANet addresses the challenge of feature extraction caused by small target sizes and diverse poses of persons in UAV remote sensing images through an effective architecture design with minimal additional parameters and computational cost. The lightweight architecture and high detection accuracy greatly enhance the potential of PTANet for real-world person detection in UAV scenarios. Drone-based target detection using visible and thermal (RGB-T) images is critical in disaster rescue, intelligent transportation, and wildlife monitoring. However, persons typically occupy fewer pixels and exhibit more varied postures than vehicles or large animals, making them difficult to detect in unmanned aerial vehicle (UAV) remote sensing images with complex backgrounds. We propose a novel progressive target-aware network (PTANet) for person detection using RGB-T images. A global adaptive feature fusion module (GAFFM) is designed to fuse the texture and thermal features of persons. A progressive focusing strategy is used. Specifically, we incorporate a person segmentation auxiliary branch (PSAB) during training to enhance target discrimination, while a cross-modality background mask (CMBM) is applied in the inference phase to suppress irrelevant background regions. Extensive experiments demonstrate that the proposed PTANet achieves high accuracy and generalization performance, reaching 79.5%, 47.8%, and 97.3% mean average precision (mAP)@50 on three drone-based person detection benchmarks (VTUAV-det, RGBTDronePerson, and VTSaR), with only 4.72 M parameters. PTANet deployed on an embedded edge device with TensorRT acceleration and quantization achieves an inference speed of 11.177 ms (640 × 640 pixels), indicating its promising potential for real-time onboard person detection. The source code is publicly available on GitHub.