LoopSparseGS: Loop Based Sparse-View Friendly Gaussian Splatting
This addresses the challenge of photorealistic 3D reconstruction from limited camera views, which is crucial for applications like virtual reality and robotics, though it is an incremental improvement over existing methods.
The paper tackles the problem of 3D Gaussian splatting's poor performance with sparse input views by proposing LoopSparseGS, which improves novel view synthesis quality through iterative point cloud densification and depth-based regularization, achieving state-of-the-art results across multiple datasets.
Despite the photorealistic novel view synthesis (NVS) performance achieved by the original 3D Gaussian splatting (3DGS), its rendering quality significantly degrades with sparse input views. This performance drop is mainly caused by the limited number of initial points generated from the sparse input, insufficient supervision during the training process, and inadequate regularization of the oversized Gaussian ellipsoids. To handle these issues, we propose the LoopSparseGS, a loop-based 3DGS framework for the sparse novel view synthesis task. In specific, we propose a loop-based Progressive Gaussian Initialization (PGI) strategy that could iteratively densify the initialized point cloud using the rendered pseudo images during the training process. Then, the sparse and reliable depth from the Structure from Motion, and the window-based dense monocular depth are leveraged to provide precise geometric supervision via the proposed Depth-alignment Regularization (DAR). Additionally, we introduce a novel Sparse-friendly Sampling (SFS) strategy to handle oversized Gaussian ellipsoids leading to large pixel errors. Comprehensive experiments on four datasets demonstrate that LoopSparseGS outperforms existing state-of-the-art methods for sparse-input novel view synthesis, across indoor, outdoor, and object-level scenes with various image resolutions.