ODE-GS: Latent ODEs for Dynamic Scene Extrapolation with 3D Gaussian Splatting
This addresses the limitation of existing dynamic scene reconstruction methods that are restricted to interpolation within fixed time windows, enabling rendering at arbitrary future timestamps for applications like simulation and visualization.
The paper tackles the problem of future extrapolation of dynamic 3D scenes by integrating 3D Gaussian Splatting with latent neural ODEs, achieving state-of-the-art extrapolation performance with a 19.8% improvement over leading baselines on benchmarks.
We introduce ODE-GS, a novel approach that integrates 3D Gaussian Splatting with latent neural ordinary differential equations (ODEs) to enable future extrapolation of dynamic 3D scenes. Unlike existing dynamic scene reconstruction methods, which rely on time-conditioned deformation networks and are limited to interpolation within a fixed time window, ODE-GS eliminates timestamp dependency by modeling Gaussian parameter trajectories as continuous-time latent dynamics. Our approach first learns an interpolation model to generate accurate Gaussian trajectories within the observed window, then trains a Transformer encoder to aggregate past trajectories into a latent state evolved via a neural ODE. Finally, numerical integration produces smooth, physically plausible future Gaussian trajectories, enabling rendering at arbitrary future timestamps. On the D-NeRF, NVFi, and HyperNeRF benchmarks, ODE-GS achieves state-of-the-art extrapolation performance, improving metrics by 19.8% compared to leading baselines, demonstrating its ability to accurately represent and predict 3D scene dynamics.