FFTransMOT: Feature-Fused Transformer for Enhanced Multi-Object Tracking

Abstract

In the field of computer vision, multi-object tracking (MOT) is a crucial task. It involves the identification, tracking, and classification of multiple objects in videos, connecting their trajectories to form a complete motion sequence. MOT comprises two core components: object detection and data association. This entails detecting objects in each frame, determining the objects to be tracked, performing data association with the next frame, and predicting the future trajectories of the objects. In this paper, we propose a model named Feature-Fused Transformer for Enhanced Multi-object Tracking (FFTransMOT). In the FFTransMOT framework, a feature fusion module is integral to synthesizing a robust representation of object features by combining information from the current and previous frames. This fusion process strengthens the feature set, enhancing its reliability for the decoder's subsequent data association tasks. The decoder leverages these improved features to accurately match objects across frames, significantly enhancing the model's tracking capabilities over time. Subsequently, the decoder conducts data association matching between frame t and the newly fused features. Additionally, we employ a self-attention mechanism to capture dependencies between input features, thereby enhancing the accuracy and stability of object detection. To validate the performance of our proposed FFTransMOT model, we conducted rigorous evaluations on four datasets (MOT16, MOT17, DanceTrack, BDD 100k). The experimental results demonstrate that the FFTransMOT model outperforms other trackers in terms of tracking accuracy and robustness in MOT tasks.