Beyond Existance: Fulfill 3D Reconstructed Scenes with Pseudo Details
This work addresses a specific bottleneck in 3D reconstruction for applications requiring high-fidelity zoomed-in views, representing an incremental improvement over existing methods.
The paper tackles the problem of training sampling deficiency in 3D Gaussian Splatting, which causes unregulated and distorted Gaussian primitives in zoomed-in views, by introducing a new training method that integrates diffusion models and multi-scale training with pseudo-ground-truth data, achieving state-of-the-art performance across various benchmarks and enriching scenes with precise details beyond training datasets.
The emergence of 3D Gaussian Splatting (3D-GS) has significantly advanced 3D reconstruction by providing high fidelity and fast training speeds across various scenarios. While recent efforts have mainly focused on improving model structures to compress data volume or reduce artifacts during zoom-in and zoom-out operations, they often overlook an underlying issue: training sampling deficiency. In zoomed-in views, Gaussian primitives can appear unregulated and distorted due to their dilation limitations and the insufficient availability of scale-specific training samples. Consequently, incorporating pseudo-details that ensure the completeness and alignment of the scene becomes essential. In this paper, we introduce a new training method that integrates diffusion models and multi-scale training using pseudo-ground-truth data. This approach not only notably mitigates the dilation and zoomed-in artifacts but also enriches reconstructed scenes with precise details out of existing scenarios. Our method achieves state-of-the-art performance across various benchmarks and extends the capabilities of 3D reconstruction beyond training datasets.