Sparfels: Fast Reconstruction from Sparse Unposed Imagery
This addresses the problem of 3D reconstruction from sparse, uncalibrated images for computer vision applications, representing an incremental improvement over existing methods.
The paper tackles sparse view reconstruction from unposed imagery by proposing an efficient pipeline that uses a 3D foundation model to initialize and guide a 2D Gaussian Splatting model, achieving state-of-the-art performance in reconstruction and novel view benchmarks on established datasets.
We present a method for Sparse view reconstruction with surface element splatting that runs within 3 minutes on a consumer grade GPU. While few methods address sparse radiance field learning from noisy or unposed sparse cameras, shape recovery remains relatively underexplored in this setting. Several radiance and shape learning test-time optimization methods address the sparse posed setting by learning data priors or using combinations of external monocular geometry priors. Differently, we propose an efficient and simple pipeline harnessing a single recent 3D foundation model. We leverage its various task heads, notably point maps and camera initializations to instantiate a bundle adjusting 2D Gaussian Splatting (2DGS) model, and image correspondences to guide camera optimization midst 2DGS training. Key to our contribution is a novel formulation of splatted color variance along rays, which can be computed efficiently. Reducing this moment in training leads to more accurate shape reconstructions. We demonstrate state-of-the-art performances in the sparse uncalibrated setting in reconstruction and novel view benchmarks based on established multi-view datasets.