Gesplat: Robust Pose-Free 3D Reconstruction via Geometry-Guided Gaussian Splatting
This work addresses the limitation of existing methods like NeRF and 3DGS that require accurate camera poses and dense views, making it more applicable in sparse-view settings for 3D reconstruction tasks.
The paper tackles the problem of 3D reconstruction and novel view synthesis from unposed sparse images, which is challenging due to unreliable camera poses and insufficient supervision, and introduces Gesplat, a framework that achieves robust performance on forward-facing and large-scale datasets compared to other pose-free methods.
Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have advanced 3D reconstruction and novel view synthesis, but remain heavily dependent on accurate camera poses and dense viewpoint coverage. These requirements limit their applicability in sparse-view settings, where pose estimation becomes unreliable and supervision is insufficient. To overcome these challenges, we introduce Gesplat, a 3DGS-based framework that enables robust novel view synthesis and geometrically consistent reconstruction from unposed sparse images. Unlike prior works that rely on COLMAP for sparse point cloud initialization, we leverage the VGGT foundation model to obtain more reliable initial poses and dense point clouds. Our approach integrates several key innovations: 1) a hybrid Gaussian representation with dual position-shape optimization enhanced by inter-view matching consistency; 2) a graph-guided attribute refinement module to enhance scene details; and 3) flow-based depth regularization that improves depth estimation accuracy for more effective supervision. Comprehensive quantitative and qualitative experiments demonstrate that our approach achieves more robust performance on both forward-facing and large-scale complex datasets compared to other pose-free methods.