Improved BOTSORT multi-object tracking algorithm for robotic rollers using feature-level fusion of millimeter-wave radar and camera sensors

Abstract

The operational safety of robotic rollers is of paramount importance, particularly in the challenging construction environment of dam construction sites. However, factors like low-illumination and intense vehicle vibrations can critically impair obstacle tracking and decision-making processes. To address this issue, this study proposes an improved BOTSORT multi-object tracking algorithm using feature-level fusion of millimeter-wave radar and camera sensors. Initially, by utilizing convolutional and PS-ROI align networks, radar and camera data are merged into feature maps, which are then processed by the improved BOTSORT algorithm using YOLOv8 instead of YOLOX for precise obstacle detection in low-illumination conditions. Additionally, an unscented Kalman filter module is employed to predict nonlinear motion of objects within the image during vibrations, while radar data refines the target association process, improving tracking accuracy under severe vibration conditions. A case study of a large-scale hydropower project demonstrates that the proposed method achieves 61.7 % mAP and 76.5 % MOTA, outperforming other obstacle detection and multi-object tracking algorithms. The proposed method improves the safety and reliability of robotic rollers under low-illumination and severe vibration working conditions.