CC-3DT: Panoramic 3D Object Tracking via Cross-Camera Fusion
This addresses the challenge of panoramic 3D tracking for autonomous vehicles, offering a significant but incremental improvement over existing camera-based methods.
The paper tackles the problem of 3D object tracking in autonomous vehicles by proposing CC-3DT, a method that fuses detections from multiple cameras before association, which reduces identity switches and improves tracking consistency, resulting in a 12.6% improvement in average multi-object tracking accuracy on the NuScenes benchmark.
To track the 3D locations and trajectories of the other traffic participants at any given time, modern autonomous vehicles are equipped with multiple cameras that cover the vehicle's full surroundings. Yet, camera-based 3D object tracking methods prioritize optimizing the single-camera setup and resort to post-hoc fusion in a multi-camera setup. In this paper, we propose a method for panoramic 3D object tracking, called CC-3DT, that associates and models object trajectories both temporally and across views, and improves the overall tracking consistency. In particular, our method fuses 3D detections from multiple cameras before association, reducing identity switches significantly and improving motion modeling. Our experiments on large-scale driving datasets show that fusion before association leads to a large margin of improvement over post-hoc fusion. We set a new state-of-the-art with 12.6% improvement in average multi-object tracking accuracy (AMOTA) among all camera-based methods on the competitive NuScenes 3D tracking benchmark, outperforming previously published methods by 6.5% in AMOTA with the same 3D detector.