Modeling Uncertainty in 3D Gaussian Splatting through Continuous Semantic Splatting
This addresses safety concerns in robotics by providing uncertainty quantification for 3D scene understanding, though it builds incrementally on existing probabilistic mapping methods.
The paper tackles the problem of 3D Gaussian Splatting failing without warning in safety-critical robotic applications by developing a novel algorithm that probabilistically updates and rasterizes semantic maps within 3D-GS, enabling per-pixel segmentation predictions with quantifiable uncertainty.
In this paper, we present a novel algorithm for probabilistically updating and rasterizing semantic maps within 3D Gaussian Splatting (3D-GS). Although previous methods have introduced algorithms which learn to rasterize features in 3D-GS for enhanced scene understanding, 3D-GS can fail without warning which presents a challenge for safety-critical robotic applications. To address this gap, we propose a method which advances the literature of continuous semantic mapping from voxels to ellipsoids, combining the precise structure of 3D-GS with the ability to quantify uncertainty of probabilistic robotic maps. Given a set of images, our algorithm performs a probabilistic semantic update directly on the 3D ellipsoids to obtain an expectation and variance through the use of conjugate priors. We also propose a probabilistic rasterization which returns per-pixel segmentation predictions with quantifiable uncertainty. We compare our method with similar probabilistic voxel-based methods to verify our extension to 3D ellipsoids, and perform ablation studies on uncertainty quantification and temporal smoothing.