Local Metrics for Multi-Object Tracking
This work addresses the need for more diagnostic evaluation metrics in MOT, particularly for applications where imperfect association is tolerable, though it is incremental as it builds on existing metrics.
The paper tackles the problem of evaluating Multi-Object Tracking (MOT) systems by introducing temporally local metrics, such as ALTA, which provide insight into trackers' ability to maintain identity over time and better identify advances in association independent of detection, as demonstrated on MOT 2017 and Waymo Open Dataset benchmarks.
This paper introduces temporally local metrics for Multi-Object Tracking. These metrics are obtained by restricting existing metrics based on track matching to a finite temporal horizon, and provide new insight into the ability of trackers to maintain identity over time. Moreover, the horizon parameter offers a novel, meaningful mechanism by which to define the relative importance of detection and association, a common dilemma in applications where imperfect association is tolerable. It is shown that the historical Average Tracking Accuracy (ATA) metric exhibits superior sensitivity to association, enabling its proposed local variant, ALTA, to capture a wide range of characteristics. In particular, ALTA is better equipped to identify advances in association independent of detection. The paper further presents an error decomposition for ATA that reveals the impact of four distinct error types and is equally applicable to ALTA. The diagnostic capabilities of ALTA are demonstrated on the MOT 2017 and Waymo Open Dataset benchmarks.