DDStereo: Efficient Dual Decoder Transformers for Stereo 3D Road Anomaly Detection
This work addresses the critical need for real-time and open-set stereo 3D detection in autonomous driving, a domain where prior stereo methods were too slow and open-set detection was unexplored.
DDStereo introduces a dual-decoder stereo transformer for real-time open-set 3D object detection, achieving state-of-the-art accuracy on public benchmarks while surpassing existing stereo methods in inference speed and matching monocular real-time performance.
Stereo-based 3D object detection still faces two critical safety challenges: real-time performance and open-set generalization. Existing stereo 3D methods typically achieve twice the accuracy of monocular methods but suffer from significantly lower inference speeds, making them unsuitable for real-time applications. Meanwhile, recent advances in open-world detection have introduced open-set and open-vocabulary algorithms in monocular 2D and 3D settings, yet stereo-based open-set detection remains largely unexplored. To bridge this gap, we propose DDStereo, a novel Dual-Decoder Stereo Transformer for real-time open-set 3D object detection. DDStereo features two lightweight decoder branches: one for open-set foreground 2D detection and the other for 3D attribute regression. These decoders share object-level queries to achieve unified target-level alignment. To enhance inference efficiency, we designed a compact disparity feature extractor and a streamlined decoder architecture. Experiments on public stereo 3D benchmarks demonstrate that DDStereo achieves state-of-the-art accuracy under both closed-set and open-set protocols. Notably, our method surpasses existing stereo 3D detectors in inference speed and, for the first time, achieves real-time performance comparable to monocular approaches.