OG-Gaussian: Occupancy Based Street Gaussians for Autonomous Driving
This reduces costs and annotation needs for creating realistic driving simulations, but is incremental as it builds on existing 3D Gaussian Splatting methods.
The paper tackles the problem of reconstructing 3D scenes for autonomous driving simulation by proposing OG-Gaussian, which uses occupancy grids from camera images instead of LiDAR to separate dynamic and static objects, achieving an average PSNR of 35.13 and 143 FPS rendering speed.
Accurate and realistic 3D scene reconstruction enables the lifelike creation of autonomous driving simulation environments. With advancements in 3D Gaussian Splatting (3DGS), previous studies have applied it to reconstruct complex dynamic driving scenes. These methods typically require expensive LiDAR sensors and pre-annotated datasets of dynamic objects. To address these challenges, we propose OG-Gaussian, a novel approach that replaces LiDAR point clouds with Occupancy Grids (OGs) generated from surround-view camera images using Occupancy Prediction Network (ONet). Our method leverages the semantic information in OGs to separate dynamic vehicles from static street background, converting these grids into two distinct sets of initial point clouds for reconstructing both static and dynamic objects. Additionally, we estimate the trajectories and poses of dynamic objects through a learning-based approach, eliminating the need for complex manual annotations. Experiments on Waymo Open dataset demonstrate that OG-Gaussian is on par with the current state-of-the-art in terms of reconstruction quality and rendering speed, achieving an average PSNR of 35.13 and a rendering speed of 143 FPS, while significantly reducing computational costs and economic overhead.