Geometry-Consistent 4D Gaussian Splatting for Sparse-Input Dynamic View Synthesis
This addresses the practical limitation of sparse-input dynamic view synthesis for AIoT applications like digital twins, offering an incremental improvement over existing methods.
The paper tackles the problem of dynamic view synthesis from sparse input views, where existing Gaussian Splatting methods degrade significantly, by introducing GC-4DGS, a framework that infuses geometric consistency into 4D Gaussian Splatting. The method outperforms RF-DeRF and original 4DGS by 2.62dB and 1.58dB in PSNR, respectively, while maintaining real-time rendering on resource-constrained IoT devices.
Gaussian Splatting has been considered as a novel way for view synthesis of dynamic scenes, which shows great potential in AIoT applications such as digital twins. However, recent dynamic Gaussian Splatting methods significantly degrade when only sparse input views are available, limiting their applicability in practice. The issue arises from the incoherent learning of 4D geometry as input views decrease. This paper presents GC-4DGS, a novel framework that infuses geometric consistency into 4D Gaussian Splatting (4DGS), offering real-time and high-quality dynamic scene rendering from sparse input views. While learning-based Multi-View Stereo (MVS) and monocular depth estimators (MDEs) provide geometry priors, directly integrating these with 4DGS yields suboptimal results due to the ill-posed nature of sparse-input 4D geometric optimization. To address these problems, we introduce a dynamic consistency checking strategy to reduce estimation uncertainties of MVS across spacetime. Furthermore, we propose a global-local depth regularization approach to distill spatiotemporal-consistent geometric information from monocular depths, thereby enhancing the coherent geometry and appearance learning within the 4D volume. Extensive experiments on the popular N3DV and Technicolor datasets validate the effectiveness of GC-4DGS in rendering quality without sacrificing efficiency. Notably, our method outperforms RF-DeRF, the latest dynamic radiance field tailored for sparse-input dynamic view synthesis, and the original 4DGS by 2.62dB and 1.58dB in PSNR, respectively, with seamless deployability on resource-constrained IoT edge devices.