Extreme Views: 3DGS Filter for Novel View Synthesis from Out-of-Distribution Camera Poses
This addresses a specific issue for 3D reconstruction systems, enabling high-fidelity navigation outside training viewpoints, but it is incremental as it builds on existing 3DGS methods.
The paper tackles the problem of visual noise in 3D Gaussian Splatting models when viewed from out-of-distribution camera poses, proposing a real-time render-aware filtering method that improves visual quality and realism compared to existing approaches like BayesRays.
When viewing a 3D Gaussian Splatting (3DGS) model from camera positions significantly outside the training data distribution, substantial visual noise commonly occurs. These artifacts result from the lack of training data in these extrapolated regions, leading to uncertain density, color, and geometry predictions from the model. To address this issue, we propose a novel real-time render-aware filtering method. Our approach leverages sensitivity scores derived from intermediate gradients, explicitly targeting instabilities caused by anisotropic orientations rather than isotropic variance. This filtering method directly addresses the core issue of generative uncertainty, allowing 3D reconstruction systems to maintain high visual fidelity even when users freely navigate outside the original training viewpoints. Experimental evaluation demonstrates that our method substantially improves visual quality, realism, and consistency compared to existing Neural Radiance Field (NeRF)-based approaches such as BayesRays. Critically, our filter seamlessly integrates into existing 3DGS rendering pipelines in real-time, unlike methods that require extensive post-hoc retraining or fine-tuning. Code and results at https://damian-bowness.github.io/EV3DGS