Unifying Short and Long-Term Tracking with Graph Hierarchies
This addresses the challenge of scalable and general multi-object tracking for video analysis, reducing the need for engineering-heavy hybrid approaches.
The paper tackles the problem of tracking objects over long videos by unifying short-term and long-term association into a single model, SUSHI, which uses graph neural networks on a hierarchy of subclips, achieving significant improvements over state-of-the-art on four diverse datasets.
Tracking objects over long videos effectively means solving a spectrum of problems, from short-term association for un-occluded objects to long-term association for objects that are occluded and then reappear in the scene. Methods tackling these two tasks are often disjoint and crafted for specific scenarios, and top-performing approaches are often a mix of techniques, which yields engineering-heavy solutions that lack generality. In this work, we question the need for hybrid approaches and introduce SUSHI, a unified and scalable multi-object tracker. Our approach processes long clips by splitting them into a hierarchy of subclips, which enables high scalability. We leverage graph neural networks to process all levels of the hierarchy, which makes our model unified across temporal scales and highly general. As a result, we obtain significant improvements over state-of-the-art on four diverse datasets. Our code and models are available at bit.ly/sushi-mot.