GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting
This addresses the problem of reconstructing 3D objects from single images for applications in computer vision and graphics, representing an incremental improvement with novel components like a pruning metric.
The paper tackles single-view 3D reconstruction by proposing GeoGS3D, a two-stage framework that uses geometric-aware diffusion and Gaussian splatting to generate multi-view consistent images and reconstruct detailed 3D objects, achieving high-quality results in experiments.
We introduce GeoGS3D, a novel two-stage framework for reconstructing detailed 3D objects from single-view images. Inspired by the success of pre-trained 2D diffusion models, our method incorporates an orthogonal plane decomposition mechanism to extract 3D geometric features from the 2D input, facilitating the generation of multi-view consistent images. During the following Gaussian Splatting, these images are fused with epipolar attention, fully utilizing the geometric correlations across views. Moreover, we propose a novel metric, Gaussian Divergence Significance (GDS), to prune unnecessary operations during optimization, significantly accelerating the reconstruction process. Extensive experiments demonstrate that GeoGS3D generates images with high consistency across views and reconstructs high-quality 3D objects, both qualitatively and quantitatively.