Relaxing Accurate Initialization Constraint for 3D Gaussian Splatting
This addresses a robustness issue for 3D reconstruction and novel view synthesis applications, making the method more practical in real-world scenarios where accurate initialization is unavailable, though it is incremental as it builds directly on 3DGS.
The paper tackles the problem of 3D Gaussian splatting's heavy dependence on accurate initialization from Structure-from-Motion, which causes performance drops with noisy or random point clouds, and proposes RAIN-GS, an optimization strategy that enables training from sub-optimal point clouds, achieving performance on-par or better than the original method with accurate initialization on multiple datasets.
3D Gaussian splatting (3DGS) has recently demonstrated impressive capabilities in real-time novel view synthesis and 3D reconstruction. However, 3DGS heavily depends on the accurate initialization derived from Structure-from-Motion (SfM) methods. When the quality of the initial point cloud deteriorates, such as in the presence of noise or when using randomly initialized point cloud, 3DGS often undergoes large performance drops. To address this limitation, we propose a novel optimization strategy dubbed RAIN-GS (Relaing Accurate Initialization Constraint for 3D Gaussian Splatting). Our approach is based on an in-depth analysis of the original 3DGS optimization scheme and the analysis of the SfM initialization in the frequency domain. Leveraging simple modifications based on our analyses, RAIN-GS successfully trains 3D Gaussians from sub-optimal point cloud (e.g., randomly initialized point cloud), effectively relaxing the need for accurate initialization. We demonstrate the efficacy of our strategy through quantitative and qualitative comparisons on multiple datasets, where RAIN-GS trained with random point cloud achieves performance on-par with or even better than 3DGS trained with accurate SfM point cloud. Our project page and code can be found at https://ku-cvlab.github.io/RAIN-GS.