Modeling uncertainty for Gaussian Splatting
This work addresses the problem of unreliable synthesized views for practitioners in computer vision, providing a method to assess confidence in outputs, though it is incremental as it builds on existing Gaussian Splatting techniques.
The paper tackles the lack of uncertainty estimation in Gaussian Splatting for novel-view synthesis by introducing Stochastic Gaussian Splatting, which integrates variational inference and a new loss term to optimize uncertainty prediction, achieving improved image rendering quality and uncertainty accuracy on the LLFF dataset.
We present Stochastic Gaussian Splatting (SGS): the first framework for uncertainty estimation using Gaussian Splatting (GS). GS recently advanced the novel-view synthesis field by achieving impressive reconstruction quality at a fraction of the computational cost of Neural Radiance Fields (NeRF). However, contrary to the latter, it still lacks the ability to provide information about the confidence associated with their outputs. To address this limitation, in this paper, we introduce a Variational Inference-based approach that seamlessly integrates uncertainty prediction into the common rendering pipeline of GS. Additionally, we introduce the Area Under Sparsification Error (AUSE) as a new term in the loss function, enabling optimization of uncertainty estimation alongside image reconstruction. Experimental results on the LLFF dataset demonstrate that our method outperforms existing approaches in terms of both image rendering quality and uncertainty estimation accuracy. Overall, our framework equips practitioners with valuable insights into the reliability of synthesized views, facilitating safer decision-making in real-world applications.