4DRC-OCC: Robust Semantic Occupancy Prediction Through Fusion of 4D Radar and Camera
This work aims to improve the robustness of 3D semantic occupancy prediction for autonomous vehicles, especially in adverse weather conditions, by introducing a novel sensor fusion approach.
This paper addresses the challenge of 3D semantic occupancy prediction in autonomous driving, particularly under adverse weather, by fusing 4D radar and camera data. The approach leverages 4D radar for robust range, velocity, and angle measurements and cameras for semantic information, further enhanced by integrating camera depth cues for improved 3D scene reconstruction. The authors also introduce an automatically labeled dataset for training.
Autonomous driving requires robust perception across diverse environmental conditions, yet 3D semantic occupancy prediction remains challenging under adverse weather and lighting. In this work, we present the first study combining 4D radar and camera data for 3D semantic occupancy prediction. Our fusion leverages the complementary strengths of both modalities: 4D radar provides reliable range, velocity, and angle measurements in challenging conditions, while cameras contribute rich semantic and texture information. We further show that integrating depth cues from camera pixels enables lifting 2D images to 3D, improving scene reconstruction accuracy. Additionally, we introduce a fully automatically labeled dataset for training semantic occupancy models, substantially reducing reliance on costly manual annotation. Experiments demonstrate the robustness of 4D radar across diverse scenarios, highlighting its potential to advance autonomous vehicle perception.