Manifold Sampling for Differentiable Uncertainty in Radiance Fields
This work addresses uncertainty estimation in radiance fields for applications like view planning and relighting, representing an incremental improvement with a novel sampling approach.
The paper tackles the problem of ambiguities and uncertainties in constructing radiance fields from image observations by proposing a method to learn Gaussian radiance fields with explicit, fine-grained uncertainty estimates. It demonstrates state-of-the-art performance on next-best-view planning tasks, including high-dimensional illumination planning for optimal relighting quality.
Radiance fields are powerful and, hence, popular models for representing the appearance of complex scenes. Yet, constructing them based on image observations gives rise to ambiguities and uncertainties. We propose a versatile approach for learning Gaussian radiance fields with explicit and fine-grained uncertainty estimates that impose only little additional cost compared to uncertainty-agnostic training. Our key observation is that uncertainties can be modeled as a low-dimensional manifold in the space of radiance field parameters that is highly amenable to Monte Carlo sampling. Importantly, our uncertainties are differentiable and, thus, allow for gradient-based optimization of subsequent captures that optimally reduce ambiguities. We demonstrate state-of-the-art performance on next-best-view planning tasks, including high-dimensional illumination planning for optimal radiance field relighting quality.