Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstruction
This addresses the challenge of generating high-quality 3D reconstructions from limited views for applications in computer vision and graphics, though it is incremental as it builds on existing Gaussian-based methods.
The paper tackles the problem of multi-view 3D reconstruction by predicting non-grid-aligned 3D Gaussians from as few as four images, using flow matching and a generative formulation to synthesize plausible content in unobserved regions, resulting in consistent improvements over prior methods with significantly fewer Gaussians and large gains in handling unobserved parts.
We present Free-Range Gaussians, a multi-view reconstruction method that predicts non-pixel, non-voxel-aligned 3D Gaussians from as few as four images. This is done through flow matching over Gaussian parameters. Our generative formulation of reconstruction allows the model to be supervised with non-grid-aligned 3D data, and enables it to synthesize plausible content in unobserved regions. Thus, it improves on prior methods that produce highly redundant grid-aligned Gaussians, and suffer from holes or blurry conditional means in unobserved regions. To handle the number of Gaussians needed for high-quality results, we introduce a hierarchical patching scheme to group spatially related Gaussians into joint transformer tokens, halving the sequence length while preserving structure. We further propose a timestep-weighted rendering loss during training, and photometric gradient guidance and classifier-free guidance at inference to improve fidelity. Experiments on Objaverse and Google Scanned Objects show consistent improvements over pixel and voxel-aligned methods while using significantly fewer Gaussians, with large gains when input views leave parts of the object unobserved.