OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Prediction
This addresses the challenge of reliable 3D scene understanding for autonomous vehicles, especially under varying lighting and weather conditions, representing an incremental improvement over existing methods.
The paper tackles the problem of 3D semantic occupancy prediction in autonomous vehicles by introducing OccFusion, a sensor fusion framework that integrates lidar and radar with camera images to improve accuracy and robustness, achieving top-tier performance on the nuScenes benchmark.
A comprehensive understanding of 3D scenes is crucial in autonomous vehicles (AVs), and recent models for 3D semantic occupancy prediction have successfully addressed the challenge of describing real-world objects with varied shapes and classes. However, existing methods for 3D occupancy prediction heavily rely on surround-view camera images, making them susceptible to changes in lighting and weather conditions. This paper introduces OccFusion, a novel sensor fusion framework for predicting 3D occupancy. By integrating features from additional sensors, such as lidar and surround view radars, our framework enhances the accuracy and robustness of occupancy prediction, resulting in top-tier performance on the nuScenes benchmark. Furthermore, extensive experiments conducted on the nuScenes and semanticKITTI dataset, including challenging night and rainy scenarios, confirm the superior performance of our sensor fusion strategy across various perception ranges. The code for this framework will be made available at https://github.com/DanielMing123/OccFusion.