Thermalizer: Stable autoregressive neural emulation of spatiotemporal chaos
This addresses the fundamental issue of error accumulation in neural emulators for spatiotemporal chaos, enabling more reliable long-term predictions in science and engineering applications.
The paper tackled the problem of autoregressive surrogate models for spatiotemporal systems accumulating errors and diverging over long rollouts, and showed that using diffusion models to estimate the invariant measure's score for on-the-fly denoising (thermalization) extends stable prediction horizons by an order of magnitude in turbulent and chaotic systems.
Autoregressive surrogate models (or \textit{emulators}) of spatiotemporal systems provide an avenue for fast, approximate predictions, with broad applications across science and engineering. At inference time, however, these models are generally unable to provide predictions over long time rollouts due to accumulation of errors leading to diverging trajectories. In essence, emulators operate out of distribution, and controlling the online distribution quickly becomes intractable in large-scale settings. To address this fundamental issue, and focusing on time-stationary systems admitting an invariant measure, we leverage diffusion models to obtain an implicit estimator of the score of this invariant measure. We show that this model of the score function can be used to stabilize autoregressive emulator rollouts by applying on-the-fly denoising during inference, a process we call \textit{thermalization}. Thermalizing an emulator rollout is shown to extend the time horizon of stable predictions by an order of magnitude in complex systems exhibiting turbulent and chaotic behavior, opening up a novel application of diffusion models in the context of neural emulation.