Uncertainty Estimation for Novel Views in Gaussian Splatting from Primitive-Based Representations of Error and Visibility
This addresses the need for reliable uncertainty estimation in Gaussian Splatting for critical applications like robotics and medicine, representing an incremental improvement over existing variance-based methods.
The paper tackles uncertainty estimation in Gaussian Splatting for novel views by establishing primitive-based representations of error and visibility from training data, which are rendered into uncertainty feature maps and aggregated via pixel-wise regression. The method outperforms state-of-the-art approaches with high correlation to true errors, particularly on foreground objects, and generalizes to new scenes without requiring holdout data.
In this work, we present a novel method for uncertainty estimation (UE) in Gaussian Splatting. UE is crucial for using Gaussian Splatting in critical applications such as robotics and medicine. Previous methods typically estimate the variance of Gaussian primitives and use the rendering process to obtain pixel-wise uncertainties. Our method establishes primitive representations of error and visibility of trainings views, which carries meaningful uncertainty information. This representation is obtained by projection of training error and visibility onto the primitives. Uncertainties of novel views are obtained by rendering the primitive representations of uncertainty for those novel views, yielding uncertainty feature maps. To aggregate these uncertainty feature maps of novel views, we perform a pixel-wise regression on holdout data. In our experiments, we analyze the different components of our method, investigating various combinations of uncertainty feature maps and regression models. Furthermore, we considered the effect of separating splatting into foreground and background. Our UEs show high correlations to true errors, outperforming state-of-the-art methods, especially on foreground objects. The trained regression models show generalization capabilities to new scenes, allowing uncertainty estimation without the need for holdout data.