Metropolis-Hastings Sampling for 3D Gaussian Reconstruction
This addresses the problem of inefficient and heuristic-dependent 3D reconstruction for computer vision researchers and practitioners, offering a more flexible and adaptive approach, though it appears incremental as it builds directly on 3D Gaussian Splatting.
The paper tackles the problem of heuristic-based density control in 3D Gaussian Splatting, which can cause redundant computations or premature removal of useful Gaussians, by proposing an adaptive sampling framework using Metropolis-Hastings that reformulates densification and pruning as a probabilistic process. The result is a reduction in the number of Gaussians needed, achieving faster convergence while matching or modestly surpassing state-of-the-art view-synthesis quality on benchmark datasets like Mip-NeRF360, Tanks and Temples, and Deep Blending.
We propose an adaptive sampling framework for 3D Gaussian Splatting (3DGS) that leverages comprehensive multi-view photometric error signals within a unified Metropolis-Hastings approach. Vanilla 3DGS heavily relies on heuristic-based density-control mechanisms (e.g., cloning, splitting, and pruning), which can lead to redundant computations or premature removal of beneficial Gaussians. Our framework overcomes these limitations by reformulating densification and pruning as a probabilistic sampling process, dynamically inserting and relocating Gaussians based on aggregated multi-view errors and opacity scores. Guided by Bayesian acceptance tests derived from these error-based importance scores, our method substantially reduces reliance on heuristics, offers greater flexibility, and adaptively infers Gaussian distributions without requiring predefined scene complexity. Experiments on benchmark datasets, including Mip-NeRF360, Tanks and Temples and Deep Blending, show that our approach reduces the number of Gaussians needed, achieving faster convergence while matching or modestly surpassing the view-synthesis quality of state-of-the-art models.