Stable Neural Flows
This work addresses stability issues in neural ODEs for researchers in machine learning, offering incremental improvements with theoretical guarantees.
The authors tackled the instability of neural ordinary differential equations by introducing a provably stable variant that evolves on a neural network-parametrized energy functional, resulting in robustness against input perturbations and reduced computational burden for numerical solvers.
We introduce a provably stable variant of neural ordinary differential equations (neural ODEs) whose trajectories evolve on an energy functional parametrised by a neural network. Stable neural flows provide an implicit guarantee on asymptotic stability of the depth-flows, leading to robustness against input perturbations and low computational burden for the numerical solver. The learning procedure is cast as an optimal control problem, and an approximate solution is proposed based on adjoint sensivity analysis. We further introduce novel regularizers designed to ease the optimization process and speed up convergence. The proposed model class is evaluated on non-linear classification and function approximation tasks.