Error estimation for physics-informed neural networks with implicit Runge-Kutta methods
This provides a method for error estimation in physics-informed neural networks, addressing a key limitation for researchers and practitioners in fields like power system modeling, though it is incremental as it builds on existing implicit Runge-Kutta techniques.
The paper tackles the problem of estimating errors in neural network-based approximations for dynamical systems, which is difficult compared to traditional numerical methods, by using the network's predictions in a high-order implicit Runge-Kutta method to relate residuals to prediction errors, resulting in error estimates that highly correlate with actual errors and improve with higher-order methods.
The ability to accurately approximate trajectories of dynamical systems enables their analysis, prediction, and control. Neural network (NN)-based approximations have attracted significant interest due to fast evaluation with good accuracy over long integration time steps. In contrast to established numerical approximation schemes such as Runge-Kutta methods, the estimation of the error of the NN-based approximations proves to be difficult. In this work, we propose to use the NN's predictions in a high-order implicit Runge-Kutta (IRK) method. The residuals in the implicit system of equations can be related to the NN's prediction error, hence, we can provide an error estimate at several points along a trajectory. We find that this error estimate highly correlates with the NN's prediction error and that increasing the order of the IRK method improves this estimate. We demonstrate this estimation methodology for Physics-Informed Neural Network (PINNs) on the logistic equation as an illustrative example and then apply it to a four-state electric generator model that is regularly used in power system modelling.