Tangent-Space Regularization for Neural-Network Models of Dynamical Systems
This addresses data efficiency and stability issues in neural-network modeling of dynamical systems for control applications, representing an incremental improvement.
The paper tackles the problem of training neural-network models of dynamical systems without requiring large amounts of training data, by introducing tangent space regularization to leverage smoothness properties of the dynamics Jacobian, resulting in improved one-step prediction and simulation performance across different architectures.
This work introduces the concept of tangent space regularization for neural-network models of dynamical systems. The tangent space to the dynamics function of many physical systems of interest in control applications exhibits useful properties, e.g., smoothness, motivating regularization of the model Jacobian along system trajectories using assumptions on the tangent space of the dynamics. Without assumptions, large amounts of training data are required for a neural network to learn the full non-linear dynamics without overfitting. We compare different network architectures on one-step prediction and simulation performance and investigate the propensity of different architectures to learn models with correct input-output Jacobian. Furthermore, the influence of $L_2$ weight regularization on the learned Jacobian eigenvalue spectrum, and hence system stability, is investigated.