On the Error Analysis of 3D Gaussian Splatting and an Optimal Projection Strategy
This addresses a fundamental limitation in real-time neural rendering for applications like VR and graphics, though it is incremental as it builds on existing splatting methods.
The paper tackles the projection errors in 3D Gaussian Splatting caused by local affine approximations, analyzing the error function and deriving an optimal projection strategy that reduces artifacts and improves rendering realism.
3D Gaussian Splatting has garnered extensive attention and application in real-time neural rendering. Concurrently, concerns have been raised about the limitations of this technology in aspects such as point cloud storage, performance, and robustness in sparse viewpoints, leading to various improvements. However, there has been a notable lack of attention to the fundamental problem of projection errors introduced by the local affine approximation inherent in the splatting itself, and the consequential impact of these errors on the quality of photo-realistic rendering. This paper addresses the projection error function of 3D Gaussian Splatting, commencing with the residual error from the first-order Taylor expansion of the projection function. The analysis establishes a correlation between the error and the Gaussian mean position. Subsequently, leveraging function optimization theory, this paper analyzes the function's minima to provide an optimal projection strategy for Gaussian Splatting referred to Optimal Gaussian Splatting, which can accommodate a variety of camera models. Experimental validation further confirms that this projection methodology reduces artifacts, resulting in a more convincingly realistic rendering.