MOTRv3: Release-Fetch Supervision for End-to-End Multi-Object Tracking
This work addresses a key bottleneck in multi-object tracking for applications like surveillance and autonomous driving, offering an incremental improvement over prior methods.
The paper tackles the conflict between detection and association in end-to-end multi-object tracking by proposing MOTRv3, which introduces a release-fetch supervision strategy to balance label assignment, achieving impressive performance on benchmarks like MOT17 and DanceTrack.
Although end-to-end multi-object trackers like MOTR enjoy the merits of simplicity, they suffer from the conflict between detection and association seriously, resulting in unsatisfactory convergence dynamics. While MOTRv2 partly addresses this problem, it demands an additional detection network for assistance. In this work, we serve as the first to reveal that this conflict arises from the unfair label assignment between detect queries and track queries during training, where these detect queries recognize targets and track queries associate them. Based on this observation, we propose MOTRv3, which balances the label assignment process using the developed release-fetch supervision strategy. In this strategy, labels are first released for detection and gradually fetched back for association. Besides, another two strategies named pseudo label distillation and track group denoising are designed to further improve the supervision for detection and association. Without the assistance of an extra detection network during inference, MOTRv3 achieves impressive performance across diverse benchmarks, e.g., MOT17, DanceTrack.