CoR-GS: Sparse-View 3D Gaussian Splatting via Co-Regularization
This work addresses the challenge of improving 3D scene reconstruction and rendering for computer vision applications under sparse-view conditions, representing an incremental advance by building on existing 3DGS methods.
The paper tackles the problem of overfitting in 3D Gaussian Splatting (3DGS) with sparse training views, which degrades rendering quality, and proposes CoR-GS, a co-regularization method that identifies and suppresses inaccurate reconstructions using point and rendering disagreements, achieving state-of-the-art novel view synthesis quality on datasets like LLFF and Mip-NeRF360.
3D Gaussian Splatting (3DGS) creates a radiance field consisting of 3D Gaussians to represent a scene. With sparse training views, 3DGS easily suffers from overfitting, negatively impacting rendering. This paper introduces a new co-regularization perspective for improving sparse-view 3DGS. When training two 3D Gaussian radiance fields, we observe that the two radiance fields exhibit point disagreement and rendering disagreement that can unsupervisedly predict reconstruction quality, stemming from the randomness of densification implementation. We further quantify the two disagreements and demonstrate the negative correlation between them and accurate reconstruction, which allows us to identify inaccurate reconstruction without accessing ground-truth information. Based on the study, we propose CoR-GS, which identifies and suppresses inaccurate reconstruction based on the two disagreements: (1) Co-pruning considers Gaussians that exhibit high point disagreement in inaccurate positions and prunes them. (2) Pseudo-view co-regularization considers pixels that exhibit high rendering disagreement are inaccurate and suppress the disagreement. Results on LLFF, Mip-NeRF360, DTU, and Blender demonstrate that CoR-GS effectively regularizes the scene geometry, reconstructs the compact representations, and achieves state-of-the-art novel view synthesis quality under sparse training views.