Focal-PETR: Embracing Foreground for Efficient Multi-Camera 3D Object Detection
This work improves efficiency for autonomous driving systems by accelerating 3D detection, though it is incremental as it builds on existing PETR methods.
The paper tackled the problem of inefficient multi-camera 3D object detection by addressing weak correlations between foreground tokens and queries, resulting in a model that achieves leading performance on the nuScenes benchmark with 30 FPS speed and 3x fewer training hours than PETR.
The dominant multi-camera 3D detection paradigm is based on explicit 3D feature construction, which requires complicated indexing of local image-view features via 3D-to-2D projection. Other methods implicitly introduce geometric positional encoding and perform global attention (e.g., PETR) to build the relationship between image tokens and 3D objects. The 3D-to-2D perspective inconsistency and global attention lead to a weak correlation between foreground tokens and queries, resulting in slow convergence. We propose Focal-PETR with instance-guided supervision and spatial alignment module to adaptively focus object queries on discriminative foreground regions. Focal-PETR additionally introduces a down-sampling strategy to reduce the consumption of global attention. Due to the highly parallelized implementation and down-sampling strategy, our model, without depth supervision, achieves leading performance on the large-scale nuScenes benchmark and a superior speed of 30 FPS on a single RTX3090 GPU. Extensive experiments show that our method outperforms PETR while consuming 3x fewer training hours. The code will be made publicly available.