SparseGS: Real-Time 360° Sparse View Synthesis using Gaussian Splatting
This addresses the challenge of sparse view synthesis for real-time 360° rendering, though it is incremental as it builds on existing 3D Gaussian Splatting methods.
The paper tackled the problem of 3D Gaussian Splatting requiring dense training views for novel view synthesis by introducing SparseGS, which achieved high-quality reconstruction with as few as 12 and 3 input images on unbounded and forward-facing scenarios, respectively, while maintaining real-time rendering.
3D Gaussian Splatting (3DGS) has recently enabled real-time rendering of unbounded 3D scenes for novel view synthesis. However, this technique requires dense training views to accurately reconstruct 3D geometry. A limited number of input views will significantly degrade reconstruction quality, resulting in artifacts such as "floaters" and "background collapse" at unseen viewpoints. In this work, we introduce SparseGS, an efficient training pipeline designed to address the limitations of 3DGS in scenarios with sparse training views. SparseGS incorporates depth priors, novel depth rendering techniques, and a pruning heuristic to mitigate floater artifacts, alongside an Unseen Viewpoint Regularization module to alleviate background collapses. Our extensive evaluations on the Mip-NeRF360, LLFF, and DTU datasets demonstrate that SparseGS achieves high-quality reconstruction in both unbounded and forward-facing scenarios, with as few as 12 and 3 input images, respectively, while maintaining fast training and real-time rendering capabilities.