Progressive Growing of Neural ODEs
This addresses performance issues in long-term time series forecasting for applications like traffic data, but it is incremental as it builds on existing NODE methods.
The paper tackled the problem of Neural ODEs degrading in performance on real-world long-term time series with complex behaviors, and proposed a progressive learning paradigm that improved vanilla NODEs by over 64% in experiments.
Neural Ordinary Differential Equations (NODEs) have proven to be a powerful modeling tool for approximating (interpolation) and forecasting (extrapolation) irregularly sampled time series data. However, their performance degrades substantially when applied to real-world data, especially long-term data with complex behaviors (e.g., long-term trend across years, mid-term seasonality across months, and short-term local variation across days). To address the modeling of such complex data with different behaviors at different frequencies (time spans), we propose a novel progressive learning paradigm of NODEs for long-term time series forecasting. Specifically, following the principle of curriculum learning, we gradually increase the complexity of data and network capacity as training progresses. Our experiments with both synthetic data and real traffic data (PeMS Bay Area traffic data) show that our training methodology consistently improves the performance of vanilla NODEs by over 64%.