FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstruction
This addresses the challenge of camera pose acquisition for sparse-view reconstruction, benefiting 3D content creation pipelines by eliminating pose management complexity.
The paper tackles the problem of 3D reconstruction from sparse-view images without requiring precise camera poses, introducing FreeSplatter, which generates high-quality 3D Gaussians and estimates camera parameters quickly, outperforming pose-dependent models and achieving comparable pose accuracy to state-of-the-art pose-free methods.
Sparse-view reconstruction models typically require precise camera poses, yet obtaining these parameters from sparse-view images remains challenging. We introduce FreeSplatter, a scalable feed-forward framework that generates high-quality 3D Gaussians from uncalibrated sparse-view images while estimating camera parameters within seconds. Our approach employs a streamlined transformer architecture where self-attention blocks facilitate information exchange among multi-view image tokens, decoding them into pixel-aligned 3D Gaussian primitives within a unified reference frame. This representation enables both high-fidelity 3D modeling and efficient camera parameter estimation using off-the-shelf solvers. We develop two specialized variants--for object-centric and scene-level reconstruction--trained on comprehensive datasets. Remarkably, FreeSplatter outperforms several pose-dependent Large Reconstruction Models (LRMs) by a notable margin while achieving comparable or even better pose estimation accuracy compared to state-of-the-art pose-free reconstruction approach MASt3R in challenging benchmarks. Beyond technical benchmarks, FreeSplatter streamlines text/image-to-3D content creation pipelines, eliminating the complexity of camera pose management while delivering exceptional visual fidelity.