Ray Denoising: Depth-aware Hard Negative Sampling for Multi-view 3D Object Detection
This work addresses depth estimation challenges for multi-view 3D object detection systems, offering a plug-and-play module with competitive gains, though it is incremental as it builds on existing DETR-style detectors.
The paper tackles the problem of imprecise predictions in multi-view 3D object detection by introducing Ray Denoising, a method that samples hard negative examples along camera rays to improve depth-aware feature learning, resulting in a 1.9% mAP improvement over state-of-the-art methods on the NuScenes dataset.
Multi-view 3D object detection systems often struggle with generating precise predictions due to the challenges in estimating depth from images, increasing redundant and incorrect detections. Our paper presents Ray Denoising, an innovative method that enhances detection accuracy by strategically sampling along camera rays to construct hard negative examples. These examples, visually challenging to differentiate from true positives, compel the model to learn depth-aware features, thereby improving its capacity to distinguish between true and false positives. Ray Denoising is designed as a plug-and-play module, compatible with any DETR-style multi-view 3D detectors, and it only minimally increases training computational costs without affecting inference speed. Our comprehensive experiments, including detailed ablation studies, consistently demonstrate that Ray Denoising outperforms strong baselines across multiple datasets. It achieves a 1.9\% improvement in mean Average Precision (mAP) over the state-of-the-art StreamPETR method on the NuScenes dataset. It shows significant performance gains on the Argoverse 2 dataset, highlighting its generalization capability. The code will be available at https://github.com/LiewFeng/RayDN.