Neural Controlled Differential Equations for Online Prediction Tasks
This work addresses a major use case for recurrent networks by enabling Neural CDEs to be applied in real-time online prediction, particularly in medical monitoring, though it is incremental as it builds on existing Neural CDE frameworks.
The paper tackled the limitation of Neural Controlled Differential Equations (Neural CDEs) for online prediction tasks by introducing new interpolation schemes that satisfy theoretical conditions like measurability and smoothness, resulting in improved performance on three medical monitoring tasks, with gains over ODE benchmarks on all tasks and over SOTA non-ODE benchmarks on two tasks.
Neural controlled differential equations (Neural CDEs) are a continuous-time extension of recurrent neural networks (RNNs), achieving state-of-the-art (SOTA) performance at modelling functions of irregular time series. In order to interpret discrete data in continuous time, current implementations rely on non-causal interpolations of the data. This is fine when the whole time series is observed in advance, but means that Neural CDEs are not suitable for use in \textit{online prediction tasks}, where predictions need to be made in real-time: a major use case for recurrent networks. Here, we show how this limitation may be rectified. First, we identify several theoretical conditions that interpolation schemes for Neural CDEs should satisfy, such as boundedness and uniqueness. Second, we use these to motivate the introduction of new schemes that address these conditions, offering in particular measurability (for online prediction), and smoothness (for speed). Third, we empirically benchmark our online Neural CDE model on three continuous monitoring tasks from the MIMIC-IV medical database: we demonstrate improved performance on all tasks against ODE benchmarks, and on two of the three tasks against SOTA non-ODE benchmarks.