Graph neural network-tracker: a graph neural network-based multi-sensor fusion framework for robust unmanned aerial vehicle tracking

Abstract

Unmanned aerial vehicle (UAV) tracking is a critical task in surveillance, security, and autonomous navigation applications. In this study, we propose graph neural network-tracker (GNN-tracker), a novel GNN-based UAV tracking framework that effectively integrates graph-based spatial-temporal modelling, Transformer-based feature extraction, and multi-sensor fusion to enhance tracking robustness and accuracy. Unlike traditional tracking approaches, GNN-tracker dynamically constructs a spatiotemporal graph representation, improving identity consistency and reducing tracking errors under OCC-heavy scenarios. Experimental evaluations on optical, thermal, and fused UAV datasets demonstrate the superiority of GNN-tracker (fused) over state-of-the-art methods. The proposed model achieves multiple object tracking accuracy (MOTA) scores of 91.4% (fused), 89.1% (optical), and 86.3% (thermal), surpassing TransT by 8.9% in MOTA and 7.7% in higher order tracking accuracy (HOTA). The HOTA scores of 82.3% (fused), 80.1% (optical), and 78.7% (thermal) validate its strong object association capabilities, while its frames per second of 58.9 (fused), 56.8 (optical), and 54.3 (thermal) ensures real-time performance. Additionally, ablation studies confirm the essential role of graph-based modelling and multi-sensor fusion, with performance drops of up to 8.9% in MOTA when these components are removed. Thus, GNN-tracker (fused) offers a highly accurate, robust, and efficient UAV tracking solution, effectively addressing real-world challenges across diverse environmental conditions and multiple sensor modalities.