SLAck: Semantic, Location, and Appearance Aware Open-Vocabulary Tracking
This addresses the challenge of generalizing trackers to novel categories in large-vocabulary scenarios, representing a strong specific gain rather than an incremental improvement.
The paper tackles the problem of open-vocabulary multiple object tracking by proposing SLAck, a framework that integrates semantics, location, and appearance cues early in association, which outperforms previous state-of-the-art methods on benchmarks like open-vocabulary MOT and TAO TETA.
Open-vocabulary Multiple Object Tracking (MOT) aims to generalize trackers to novel categories not in the training set. Currently, the best-performing methods are mainly based on pure appearance matching. Due to the complexity of motion patterns in the large-vocabulary scenarios and unstable classification of the novel objects, the motion and semantics cues are either ignored or applied based on heuristics in the final matching steps by existing methods. In this paper, we present a unified framework SLAck that jointly considers semantics, location, and appearance priors in the early steps of association and learns how to integrate all valuable information through a lightweight spatial and temporal object graph. Our method eliminates complex post-processing heuristics for fusing different cues and boosts the association performance significantly for large-scale open-vocabulary tracking. Without bells and whistles, we outperform previous state-of-the-art methods for novel classes tracking on the open-vocabulary MOT and TAO TETA benchmarks. Our code is available at \href{https://github.com/siyuanliii/SLAck}{github.com/siyuanliii/SLAck}.