Neural DAEs: Constrained neural networks
This work addresses accuracy and efficiency challenges in neural network-based dynamical system modeling, though it appears incremental as it adapts existing mathematical methods to neural networks.
The paper tackles the problem of improving neural networks for dynamical systems by explicitly incorporating auxiliary algebraic trajectory information, drawing inspiration from differential-algebraic equations. The result shows that constraint methods provide significant inference boosts with limited training impact, as demonstrated in multi-body pendulum and molecular dynamics simulations.
This article investigates the effect of explicitly adding auxiliary algebraic trajectory information to neural networks for dynamical systems. We draw inspiration from the field of differential-algebraic equations and differential equations on manifolds and implement related methods in residual neural networks, despite some fundamental scenario differences. Constraint or auxiliary information effects are incorporated through stabilization as well as projection methods, and we show when to use which method based on experiments involving simulations of multi-body pendulums and molecular dynamics scenarios. Several of our methods are easy to implement in existing code and have limited impact on training performance while giving significant boosts in terms of inference.