ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every Detection Box
This work addresses object tracking challenges in computer vision for applications like autonomous driving, offering a generic and nonparametric solution that can be integrated with various detectors.
The paper tackles the problem of object missing and fragmented trajectories in multi-object tracking by proposing a hierarchical data association strategy to utilize low-score detection boxes, achieving state-of-the-art results with 56.4% AMOTA on camera and 70.1% AMOTA on LiDAR in the nuScenes 3D MOT benchmark.
Multi-object tracking (MOT) aims at estimating bounding boxes and identities of objects across video frames. Detection boxes serve as the basis of both 2D and 3D MOT. The inevitable changing of detection scores leads to object missing after tracking. We propose a hierarchical data association strategy to mine the true objects in low-score detection boxes, which alleviates the problems of object missing and fragmented trajectories. The simple and generic data association strategy shows effectiveness under both 2D and 3D settings. In 3D scenarios, it is much easier for the tracker to predict object velocities in the world coordinate. We propose a complementary motion prediction strategy that incorporates the detected velocities with a Kalman filter to address the problem of abrupt motion and short-term disappearing. ByteTrackV2 leads the nuScenes 3D MOT leaderboard in both camera (56.4% AMOTA) and LiDAR (70.1% AMOTA) modalities. Furthermore, it is nonparametric and can be integrated with various detectors, making it appealing in real applications. The source code is released at https://github.com/ifzhang/ByteTrack-V2.