StereoDETR: Stereo-based Transformer for 3D Object Detection
This work addresses the speed bottleneck in stereo 3D object detection for applications like autonomous driving, though it is incremental as it builds upon existing DETR methods.
The paper tackles the problem of high computational overhead and latency in stereo-based 3D object detection by proposing StereoDETR, an efficient framework that achieves real-time inference and surpasses monocular approaches in speed while setting new state-of-the-art results on pedestrian and cyclist subsets of the KITTI benchmark.
Compared to monocular 3D object detection, stereo-based 3D methods offer significantly higher accuracy but still suffer from high computational overhead and latency. The state-of-the-art stereo 3D detection method achieves twice the accuracy of monocular approaches, yet its inference speed is only half as fast. In this paper, we propose StereoDETR, an efficient stereo 3D object detection framework based on DETR. StereoDETR consists of two branches: a monocular DETR branch and a stereo branch. The DETR branch is built upon 2D DETR with additional channels for predicting object scale, orientation, and sampling points. The stereo branch leverages low-cost multi-scale disparity features to predict object-level depth maps. These two branches are coupled solely through a differentiable depth sampling strategy. To handle occlusion, we introduce a constrained supervision strategy for sampling points without requiring extra annotations. StereoDETR achieves real-time inference and is the first stereo-based method to surpass monocular approaches in speed. It also achieves competitive accuracy on the public KITTI benchmark, setting new state-of-the-art results on pedestrian and cyclist subsets. The code is available at https://github.com/shiyi-mu/StereoDETR-OPEN.