Efficient Training of Physics-enhanced Neural ODEs via Direct Collocation and Nonlinear Programming
This work addresses the challenge of efficiently training neural ODEs with physical constraints for applications in modeling and simulation, representing an incremental advancement over existing direct collocation methods.
The paper tackles the problem of training Physics-enhanced Neural ODEs (PeN-ODEs) by formulating it as a dynamic optimization problem using direct collocation and nonlinear programming, resulting in improved accuracy, speed, and generalization with smaller networks compared to other methods, as demonstrated on benchmarks like a Quarter Vehicle Model and a Van-der-Pol oscillator.
We propose a novel approach for training Physics-enhanced Neural ODEs (PeN-ODEs) by expressing the training process as a dynamic optimization problem. The full model, including neural components, is discretized using a high-order implicit Runge-Kutta method with flipped Legendre-Gauss-Radau points, resulting in a large-scale nonlinear program (NLP) efficiently solved by state-of-the-art NLP solvers such as Ipopt. This formulation enables simultaneous optimization of network parameters and state trajectories, addressing key limitations of ODE solver-based training in terms of stability, runtime, and accuracy. Extending on a recent direct collocation-based method for Neural ODEs, we generalize to PeN-ODEs, incorporate physical constraints, and present a custom, parallelized, open-source implementation. Benchmarks on a Quarter Vehicle Model and a Van-der-Pol oscillator demonstrate superior accuracy, speed, generalization with smaller networks compared to other training techniques. We also outline a planned integration into OpenModelica to enable accessible training of Neural DAEs.