DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems
This work addresses the problem of unstable learning in chaotic systems for researchers in fields like weather and climate modeling, though it appears incremental as it builds on existing learning objectives with a new regularization approach.
The authors tackled the challenge of learning dynamics from dissipative chaotic systems, which are prone to instability and error amplification, by proposing a framework that targets both the invariant measure and the dynamics, resulting in improved point-wise tracking and long-term statistical accuracy compared to other methods.
Learning dynamics from dissipative chaotic systems is notoriously difficult due to their inherent instability, as formalized by their positive Lyapunov exponents, which exponentially amplify errors in the learned dynamics. However, many of these systems exhibit ergodicity and an attractor: a compact and highly complex manifold, to which trajectories converge in finite-time, that supports an invariant measure, i.e., a probability distribution that is invariant under the action of the dynamics, which dictates the long-term statistical behavior of the system. In this work, we leverage this structure to propose a new framework that targets learning the invariant measure as well as the dynamics, in contrast with typical methods that only target the misfit between trajectories, which often leads to divergence as the trajectories' length increases. We use our framework to propose a tractable and sample efficient objective that can be used with any existing learning objectives. Our Dynamics Stable Learning by Invariant Measure (DySLIM) objective enables model training that achieves better point-wise tracking and long-term statistical accuracy relative to other learning objectives. By targeting the distribution with a scalable regularization term, we hope that this approach can be extended to more complex systems exhibiting slowly-variant distributions, such as weather and climate models.