RCDINO: Enhancing Radar-Camera 3D Object Detection with DINOv2 Semantic Features
This work addresses multimodal fusion for autonomous driving, offering incremental improvements in radar-camera detection.
The paper tackles 3D object detection for autonomous driving by proposing RCDINO, a model that fuses visual features with DINOv2 semantic representations, achieving state-of-the-art performance on the nuScenes dataset with 56.4 NDS and 48.1 mAP.
Three-dimensional object detection is essential for autonomous driving and robotics, relying on effective fusion of multimodal data from cameras and radar. This work proposes RCDINO, a multimodal transformer-based model that enhances visual backbone features by fusing them with semantically rich representations from the pretrained DINOv2 foundation model. This approach enriches visual representations and improves the model's detection performance while preserving compatibility with the baseline architecture. Experiments on the nuScenes dataset demonstrate that RCDINO achieves state-of-the-art performance among radar-camera models, with 56.4 NDS and 48.1 mAP. Our implementation is available at https://github.com/OlgaMatykina/RCDINO.