AM-SORT: Adaptable Motion Predictor with Historical Trajectory Embedding for Multi-Object Tracking
This addresses the problem of inaccurate object tracking in dynamic environments for applications like surveillance and autonomous driving, representing an incremental improvement over existing SORT-series trackers.
The paper tackles the limitation of Kalman Filter-based multi-object tracking in non-linear motion and occlusion scenarios by proposing AM-SORT, which replaces the Kalman Filter with a transformer-based motion predictor using historical trajectory embedding, achieving competitive performance with 56.3 IDF1 and 55.6 HOTA on DanceTrack.
Many multi-object tracking (MOT) approaches, which employ the Kalman Filter as a motion predictor, assume constant velocity and Gaussian-distributed filtering noises. These assumptions render the Kalman Filter-based trackers effective in linear motion scenarios. However, these linear assumptions serve as a key limitation when estimating future object locations within scenarios involving non-linear motion and occlusions. To address this issue, we propose a motion-based MOT approach with an adaptable motion predictor, called AM-SORT, which adapts to estimate non-linear uncertainties. AM-SORT is a novel extension of the SORT-series trackers that supersedes the Kalman Filter with the transformer architecture as a motion predictor. We introduce a historical trajectory embedding that empowers the transformer to extract spatio-temporal features from a sequence of bounding boxes. AM-SORT achieves competitive performance compared to state-of-the-art trackers on DanceTrack, with 56.3 IDF1 and 55.6 HOTA. We conduct extensive experiments to demonstrate the effectiveness of our method in predicting non-linear movement under occlusions.