From Coarse to Fine: Learnable Discrete Wavelet Transforms for Efficient 3D Gaussian Splatting
This addresses memory and bandwidth issues for 3D Gaussian Splatting users in novel view synthesis, offering an incremental improvement by integrating seamlessly with existing frameworks.
The paper tackles the problem of memory and bandwidth strain in 3D Gaussian Splatting due to an ever-growing set of Gaussian primitives, introducing AutoOpti3DGS to automatically restrain Gaussian proliferation without sacrificing visual fidelity, resulting in sparser scene representations more compatible with constrained hardware.
3D Gaussian Splatting has emerged as a powerful approach in novel view synthesis, delivering rapid training and rendering but at the cost of an ever-growing set of Gaussian primitives that strains memory and bandwidth. We introduce AutoOpti3DGS, a training-time framework that automatically restrains Gaussian proliferation without sacrificing visual fidelity. The key idea is to feed the input images to a sequence of learnable Forward and Inverse Discrete Wavelet Transforms, where low-pass filters are kept fixed, high-pass filters are learnable and initialized to zero, and an auxiliary orthogonality loss gradually activates fine frequencies. This wavelet-driven, coarse-to-fine process delays the formation of redundant fine Gaussians, allowing 3DGS to capture global structure first and refine detail only when necessary. Through extensive experiments, AutoOpti3DGS requires just a single filter learning-rate hyper-parameter, integrates seamlessly with existing efficient 3DGS frameworks, and consistently produces sparser scene representations more compatible with memory or storage-constrained hardware.