MeMOT: Multi-Object Tracking with Memory
This addresses the challenge of robust object tracking in videos for applications like surveillance and autonomous driving, but it is incremental as it builds on existing Transformer-based methods.
The paper tackles the problem of multi-object tracking by proposing an online algorithm that links objects over long time spans using a large spatio-temporal memory, achieving competitive performance on benchmark datasets.
We propose an online tracking algorithm that performs the object detection and data association under a common framework, capable of linking objects after a long time span. This is realized by preserving a large spatio-temporal memory to store the identity embeddings of the tracked objects, and by adaptively referencing and aggregating useful information from the memory as needed. Our model, called MeMOT, consists of three main modules that are all Transformer-based: 1) Hypothesis Generation that produce object proposals in the current video frame; 2) Memory Encoding that extracts the core information from the memory for each tracked object; and 3) Memory Decoding that solves the object detection and data association tasks simultaneously for multi-object tracking. When evaluated on widely adopted MOT benchmark datasets, MeMOT observes very competitive performance.