Space-Time Forecasting of Dynamic Scenes with Motion-aware Gaussian Grouping
This work addresses the problem of long-term scene forecasting for applications like robotics and autonomous systems, representing an incremental improvement over prior methods.
The paper tackles the challenge of forecasting dynamic scenes in computer vision by introducing Motion Group-aware Gaussian Forecasting (MoGaF), which uses motion-aware Gaussian grouping and a lightweight forecasting module to achieve realistic and stable long-term scene evolution, outperforming existing baselines in rendering quality, motion plausibility, and forecasting stability.
Forecasting dynamic scenes remains a fundamental challenge in computer vision, as limited observations make it difficult to capture coherent object-level motion and long-term temporal evolution. We present Motion Group-aware Gaussian Forecasting (MoGaF), a framework for long-term scene extrapolation built upon the 4D Gaussian Splatting representation. MoGaF introduces motion-aware Gaussian grouping and group-wise optimization to enforce physically consistent motion across both rigid and non-rigid regions, yielding spatially coherent dynamic representations. Leveraging this structured space-time representation, a lightweight forecasting module predicts future motion, enabling realistic and temporally stable scene evolution. Experiments on synthetic and real-world datasets demonstrate that MoGaF consistently outperforms existing baselines in rendering quality, motion plausibility, and long-term forecasting stability. Our project page is available at https://slime0519.github.io/mogaf