Multiple shooting for training neural differential equations on time series
This work addresses a specific issue in modeling oscillatory data with neural differential equations, representing an incremental improvement in training methods for time-series analysis.
The authors tackled the problem of neural differential equations failing to fit oscillatory time-series data by introducing the multiple shooting method, which successfully fitted two datasets where the standard approach failed, using penalty or augmented Lagrangian methods to satisfy constraints.
Neural differential equations have recently emerged as a flexible data-driven/hybrid approach to model time-series data. This work experimentally demonstrates that if the data contains oscillations, then standard fitting of a neural differential equation may result in a flattened out trajectory that fails to describe the data. We then introduce the multiple shooting method and present successful demonstrations of this method for the fitting of a neural differential equation to two datasets (synthetic and experimental) that the standard approach fails to fit. Constraints introduced by multiple shooting can be satisfied using a penalty or augmented Lagrangian method.