3D Video Object Detection with Learnable Object-Centric Global Optimization
This work improves 3D object detection in autonomous driving by handling moving objects more effectively, though it is incremental as it builds on existing correspondence-based methods.
The paper tackles the problem of 3D video object detection by addressing moving objects as outliers in correspondence-based optimization, proposing BA-Det with object-centric temporal correspondence learning and featuremetric object bundle adjustment. It achieves state-of-the-art performance on the Waymo Open Dataset with marginal computation cost.
We explore long-term temporal visual correspondence-based optimization for 3D video object detection in this work. Visual correspondence refers to one-to-one mappings for pixels across multiple images. Correspondence-based optimization is the cornerstone for 3D scene reconstruction but is less studied in 3D video object detection, because moving objects violate multi-view geometry constraints and are treated as outliers during scene reconstruction. We address this issue by treating objects as first-class citizens during correspondence-based optimization. In this work, we propose BA-Det, an end-to-end optimizable object detector with object-centric temporal correspondence learning and featuremetric object bundle adjustment. Empirically, we verify the effectiveness and efficiency of BA-Det for multiple baseline 3D detectors under various setups. Our BA-Det achieves SOTA performance on the large-scale Waymo Open Dataset (WOD) with only marginal computation cost. Our code is available at https://github.com/jiaweihe1996/BA-Det.