Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Models
This work addresses a gap in applying diffusion models to temporal predictive learning, benefiting researchers and practitioners in fields like video analysis and time series forecasting, though it is incremental as it builds on existing diffusion model frameworks.
The paper tackles the problem of generating temporally coherent sequences in predictive learning by introducing Dynamical Diffusion (DyDiff), a framework that incorporates temporally aware processes into diffusion models, resulting in consistent performance improvements across scientific spatiotemporal forecasting, video prediction, and time series forecasting tasks.
Diffusion models have emerged as powerful generative frameworks by progressively adding noise to data through a forward process and then reversing this process to generate realistic samples. While these models have achieved strong performance across various tasks and modalities, their application to temporal predictive learning remains underexplored. Existing approaches treat predictive learning as a conditional generation problem, but often fail to fully exploit the temporal dynamics inherent in the data, leading to challenges in generating temporally coherent sequences. To address this, we introduce Dynamical Diffusion (DyDiff), a theoretically sound framework that incorporates temporally aware forward and reverse processes. Dynamical Diffusion explicitly models temporal transitions at each diffusion step, establishing dependencies on preceding states to better capture temporal dynamics. Through the reparameterization trick, Dynamical Diffusion achieves efficient training and inference similar to any standard diffusion model. Extensive experiments across scientific spatiotemporal forecasting, video prediction, and time series forecasting demonstrate that Dynamical Diffusion consistently improves performance in temporal predictive tasks, filling a crucial gap in existing methodologies. Code is available at this repository: https://github.com/thuml/dynamical-diffusion.