SpOT: Spatiotemporal Modeling for 3D Object Tracking
This addresses the problem of consistently tracking objects in 3D scenes for applications like autonomous driving, representing an incremental improvement over existing methods.
The paper tackled 3D multi-object tracking by developing a holistic spatiotemporal representation that leverages long temporal histories of objects, achieving state-of-the-art performance on Waymo and nuScenes benchmarks.
3D multi-object tracking aims to uniquely and consistently identify all mobile entities through time. Despite the rich spatiotemporal information available in this setting, current 3D tracking methods primarily rely on abstracted information and limited history, e.g. single-frame object bounding boxes. In this work, we develop a holistic representation of traffic scenes that leverages both spatial and temporal information of the actors in the scene. Specifically, we reformulate tracking as a spatiotemporal problem by representing tracked objects as sequences of time-stamped points and bounding boxes over a long temporal history. At each timestamp, we improve the location and motion estimates of our tracked objects through learned refinement over the full sequence of object history. By considering time and space jointly, our representation naturally encodes fundamental physical priors such as object permanence and consistency across time. Our spatiotemporal tracking framework achieves state-of-the-art performance on the Waymo and nuScenes benchmarks.