3DGS$^2$-TR: Scalable Second-Order Trust-Region Method for 3D Gaussian Splatting
This work addresses the computational and memory bottlenecks in 3D scene reconstruction for applications like computer graphics and VR, offering a scalable solution that is incremental over existing second-order methods.
The paper tackles the problem of accelerating scene training in 3D Gaussian Splatting by proposing a second-order optimizer that uses diagonal Hessian approximation and a trust-region technique, achieving better reconstruction quality with 50% fewer training iterations and less than 1GB of peak GPU memory overhead compared to ADAM.
We propose 3DGS$^2$-TR,a second-order optimizer for accelerating the scene training problem in 3D Gaussian Splatting (3DGS). Unlike existing second-order approaches that rely on explicit or dense curvature representations, such as 3DGS-LM (Höllein et al., 2025) or 3DGS2 (Lan et al., 2025), our method approximates curvature using only the diagonal of the Hessian matrix, efficiently via Hutchinson's method. Our approach is fully matrix-free and has the same complexity as ADAM (Kingma, 2024), $O(n)$ in both computation and memory costs. To ensure stable optimization in the presence of strong nonlinearity in the 3DGS rasterization process, we introduce a parameter-wise trust-region technique based on the squared Hellinger distance, regularizing updates to Gaussian parameters. Under identical parameter initialization and without densification, 3DGS$^2$-TR is able to achieve better reconstruction quality on standard datasets, using 50% fewer training iterations compared to ADAM, while incurring less than 1GB of peak GPU memory overhead (17% more than ADAM and 85% less than 3DGS-LM), enabling scalability to very large scenes and potentially to distributed training settings.