When are dynamical systems learned from time series data statistically accurate?
This addresses the issue of ensuring learned models accurately reproduce physical dynamics, which is crucial for applications in physics and engineering, though it is incremental in refining generalization theory.
The paper tackles the problem that conventional generalization metrics fail to capture physical behavior in learned dynamical systems, proposing an ergodic theoretic approach to define and analyze generalization for neural representations of ergodic systems, with results showing improved statistical accuracy when adding Jacobian information during training.
Conventional notions of generalization often fail to describe the ability of learned models to capture meaningful information from dynamical data. A neural network that learns complex dynamics with a small test error may still fail to reproduce its \emph{physical} behavior, including associated statistical moments and Lyapunov exponents. To address this gap, we propose an ergodic theoretic approach to generalization of complex dynamical models learned from time series data. Our main contribution is to define and analyze generalization of a broad suite of neural representations of classes of ergodic systems, including chaotic systems, in a way that captures emulating underlying invariant, physical measures. Our results provide theoretical justification for why regression methods for generators of dynamical systems (Neural ODEs) fail to generalize, and why their statistical accuracy improves upon adding Jacobian information during training. We verify our results on a number of ergodic chaotic systems and neural network parameterizations, including MLPs, ResNets, Fourier Neural layers, and RNNs.