3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention
This work addresses the challenge of labeling 3D trajectory data at scale without high-cost human experts, primarily for the autonomous systems community, representing an incremental advance in 3D tracking methods.
The paper tackles the problem of 3D offline multi-object tracking by proposing Batch3DMOT, a method using graph neural networks with cross-edge modality attention, which achieves a 3.3% improvement in AMOTA score on nuScenes and sets a new state-of-the-art.
Online 3D multi-object tracking (MOT) has witnessed significant research interest in recent years, largely driven by demand from the autonomous systems community. However, 3D offline MOT is relatively less explored. Labeling 3D trajectory scene data at a large scale while not relying on high-cost human experts is still an open research question. In this work, we propose Batch3DMOT which follows the tracking-by-detection paradigm and represents real-world scenes as directed, acyclic, and category-disjoint tracking graphs that are attributed using various modalities such as camera, LiDAR, and radar. We present a multi-modal graph neural network that uses a cross-edge attention mechanism mitigating modality intermittence, which translates into sparsity in the graph domain. Additionally, we present attention-weighted convolutions over frame-wise k-NN neighborhoods as suitable means to allow information exchange across disconnected graph components. We evaluate our approach using various sensor modalities and model configurations on the challenging nuScenes and KITTI datasets. Extensive experiments demonstrate that our proposed approach yields an overall improvement of 3.3% in the AMOTA score on nuScenes thereby setting the new state-of-the-art for 3D tracking and further enhancing false positive filtering.