Stability-Informed Initialization of Neural Ordinary Differential Equations
This work addresses training challenges in Neural ODEs for researchers and practitioners, offering an incremental improvement through better initialization.
The paper tackles the training of Neural ODEs by analyzing how numerical integration techniques and stability regions affect performance, and introduces a stability-informed initialization method that improves results across benchmarks and industrial applications.
This paper addresses the training of Neural Ordinary Differential Equations (neural ODEs), and in particular explores the interplay between numerical integration techniques, stability regions, step size, and initialization techniques. It is shown how the choice of integration technique implicitly regularizes the learned model, and how the solver's corresponding stability region affects training and prediction performance. From this analysis, a stability-informed parameter initialization technique is introduced. The effectiveness of the initialization method is displayed across several learning benchmarks and industrial applications.