Tracking Objects as Pixel-wise Distributions
This addresses the problem of accurate object association in multi-object tracking for computer vision applications, representing a novel method rather than an incremental improvement.
The paper tackles multi-object tracking by proposing to track objects as pixel-wise distributions instead of bounding boxes or points, resulting in P3AFormer achieving 81.2% MOTA on MOT17, the first transformer network to exceed 80% MOTA, and outperforming state-of-the-art on other benchmarks.
Multi-object tracking (MOT) requires detecting and associating objects through frames. Unlike tracking via detected bounding boxes or tracking objects as points, we propose tracking objects as pixel-wise distributions. We instantiate this idea on a transformer-based architecture, P3AFormer, with pixel-wise propagation, prediction, and association. P3AFormer propagates pixel-wise features guided by flow information to pass messages between frames. Furthermore, P3AFormer adopts a meta-architecture to produce multi-scale object feature maps. During inference, a pixel-wise association procedure is proposed to recover object connections through frames based on the pixel-wise prediction. P3AFormer yields 81.2\% in terms of MOTA on the MOT17 benchmark -- the first among all transformer networks to reach 80\% MOTA in literature. P3AFormer also outperforms state-of-the-arts on the MOT20 and KITTI benchmarks.