GaussianBeV: 3D Gaussian Representation meets Perception Models for BeV Segmentation
This work addresses a key bottleneck in autonomous driving and robotics by improving scene understanding accuracy, though it is an incremental advance building on existing 3D representation techniques.
The paper tackles the problem of insufficient detail in Bird's-eye View (BeV) representations for 3D perception by proposing GaussianBeV, which uses 3D gaussians to finely represent scenes and splatters them into BeV feature maps, achieving state-of-the-art results on the nuScenes dataset for BeV semantic segmentation.
The Bird's-eye View (BeV) representation is widely used for 3D perception from multi-view camera images. It allows to merge features from different cameras into a common space, providing a unified representation of the 3D scene. The key component is the view transformer, which transforms image views into the BeV. However, actual view transformer methods based on geometry or cross-attention do not provide a sufficiently detailed representation of the scene, as they use a sub-sampling of the 3D space that is non-optimal for modeling the fine structures of the environment. In this paper, we propose GaussianBeV, a novel method for transforming image features to BeV by finely representing the scene using a set of 3D gaussians located and oriented in 3D space. This representation is then splattered to produce the BeV feature map by adapting recent advances in 3D representation rendering based on gaussian splatting. GaussianBeV is the first approach to use this 3D gaussian modeling and 3D scene rendering process online, i.e. without optimizing it on a specific scene and directly integrated into a single stage model for BeV scene understanding. Experiments show that the proposed representation is highly effective and place GaussianBeV as the new state-of-the-art on the BeV semantic segmentation task on the nuScenes dataset.