High-fidelity 3D Gaussian Inpainting: preserving multi-view consistency and photorealistic details
This addresses the challenge of maintaining multi-view consistency in 3D scene inpainting for applications in 3D content creation, though it appears incremental as it builds on existing 3D Gaussian Splatting methods.
The paper tackles the problem of inpainting 3D scenes by proposing a novel 3D Gaussian inpainting framework that reconstructs complete scenes from sparse inpainted views, achieving state-of-the-art performance in visual quality and view consistency on diverse datasets.
Recent advancements in multi-view 3D reconstruction and novel-view synthesis, particularly through Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have greatly enhanced the fidelity and efficiency of 3D content creation. However, inpainting 3D scenes remains a challenging task due to the inherent irregularity of 3D structures and the critical need for maintaining multi-view consistency. In this work, we propose a novel 3D Gaussian inpainting framework that reconstructs complete 3D scenes by leveraging sparse inpainted views. Our framework incorporates an automatic Mask Refinement Process and region-wise Uncertainty-guided Optimization. Specifically, we refine the inpainting mask using a series of operations, including Gaussian scene filtering and back-projection, enabling more accurate localization of occluded regions and realistic boundary restoration. Furthermore, our Uncertainty-guided Fine-grained Optimization strategy, which estimates the importance of each region across multi-view images during training, alleviates multi-view inconsistencies and enhances the fidelity of fine details in the inpainted results. Comprehensive experiments conducted on diverse datasets demonstrate that our approach outperforms existing state-of-the-art methods in both visual quality and view consistency.