Does 3D Gaussian Splatting Need Accurate Volumetric Rendering?
This addresses the trade-off between rendering accuracy and performance for researchers in 3D scene reconstruction and novel-view synthesis, showing incremental insights into 3DGS's robustness.
The paper investigates whether replacing approximations in 3D Gaussian Splatting (3DGS) with more accurate volumetric rendering improves quality, finding that 3DGS outperforms volumetric rendering due to efficient optimization and a large number of Gaussians, especially with many primitives.
Since its introduction, 3D Gaussian Splatting (3DGS) has become an important reference method for learning 3D representations of a captured scene, allowing real-time novel-view synthesis with high visual quality and fast training times. Neural Radiance Fields (NeRFs), which preceded 3DGS, are based on a principled ray-marching approach for volumetric rendering. In contrast, while sharing a similar image formation model with NeRF, 3DGS uses a hybrid rendering solution that builds on the strengths of volume rendering and primitive rasterization. A crucial benefit of 3DGS is its performance, achieved through a set of approximations, in many cases with respect to volumetric rendering theory. A naturally arising question is whether replacing these approximations with more principled volumetric rendering solutions can improve the quality of 3DGS. In this paper, we present an in-depth analysis of the various approximations and assumptions used by the original 3DGS solution. We demonstrate that, while more accurate volumetric rendering can help for low numbers of primitives, the power of efficient optimization and the large number of Gaussians allows 3DGS to outperform volumetric rendering despite its approximations.