$\text{VG}^2$GT: Voxel-Gaussian Splatting Visual Geometry Grounded Transformer
For 3D reconstruction and novel view synthesis, this work provides a plug-and-play feed-forward approach that reduces training cost and improves geometric accuracy over existing methods.
VG^2GT introduces a feed-forward Gaussian splatting method that uses a frozen visual foundation model and a multi-scale voxel module to achieve geometrically accurate 3D reconstruction without per-scene optimization, outperforming SOTA on DTU, Replica, TAT, and ScanNet datasets.
Gaussian splatting has shown strong potential for 3D reconstruction and novel view synthesis. However, most existing methods require accurate camera parameters and per-scene optimization, while feed-forward methods with pixel-aligned Gaussian primitives often suffer from artifacts and non-uniform primitives. In this paper, we propose $\text{VG}^2$GT, a Voxel-Gaussian Splatting Visual Geometry-Grounded Transformer. $\text{VG}^2$GT leverages a frozen pretrained visual foundation model (VFM), incorporates a multi-scale differentiable voxel module to enhance geometric understanding, and directly splits and regresses Gaussian primitive parameters from voxel features. During training, depth maps are supervised through stochastic solid volume rendering, enabling geometrically accurate Gaussian scene reconstruction while keeping the visual foundation model fully frozen. This design enables $\text{VG}^2$GT to be seamlessly plugged into any patch-feature-based VFM, while substantially reducing the required training cost. $\text{VG}^2$GT outperforms current state-of-the-art methods on widely used DTU, Replica, TAT, and ScanNet datasets.