Spectral-GS: Taming 3D Gaussian Splatting with Spectral Entropy
This addresses a specific technical limitation in 3D reconstruction for computer vision and graphics researchers, representing an incremental improvement over existing methods like Mip-Splatting.
The paper tackles needle-like artifacts in 3D Gaussian Splatting (3D-GS) for novel view synthesis, which cause overfitting and view inconsistencies, by introducing Spectral-GS with shape-aware splitting and view-consistent filtering to achieve high-quality photorealistic rendering without noticeable artifacts.
Recently, 3D Gaussian Splatting (3D-GS) has achieved impressive results in novel view synthesis, demonstrating high fidelity and efficiency. However, it easily exhibits needle-like artifacts, especially when increasing the sampling rate. Mip-Splatting tries to remove these artifacts with a 3D smoothing filter for frequency constraints and a 2D Mip filter for approximated supersampling. Unfortunately, it tends to produce over-blurred results, and sometimes needle-like Gaussians still persist. Our spectral analysis of the covariance matrix during optimization and densification reveals that current 3D-GS lacks shape awareness, relying instead on spectral radius and view positional gradients to determine splitting. As a result, needle-like Gaussians with small positional gradients and low spectral entropy fail to split and overfit high-frequency details. Furthermore, both the filters used in 3D-GS and Mip-Splatting reduce the spectral entropy and increase the condition number during zooming in to synthesize novel view, causing view inconsistencies and more pronounced artifacts. Our Spectral-GS, based on spectral analysis, introduces 3D shape-aware splitting and 2D view-consistent filtering strategies, effectively addressing these issues, enhancing 3D-GS's capability to represent high-frequency details without noticeable artifacts, and achieving high-quality photorealistic rendering.