Log Neural Controlled Differential Equations: The Lie Brackets Make a Difference
This provides a more effective and efficient approach for handling real-world time series data, though it appears incremental as it builds on prior neural rough differential equations.
The authors tackled the problem of modeling multivariate time series with irregular sampling by introducing Log-NCDEs, a method that outperformed existing models like NCDEs, NRDEs, and MAMBA on datasets with up to 50,000 observations.
The vector field of a controlled differential equation (CDE) describes the relationship between a control path and the evolution of a solution path. Neural CDEs (NCDEs) treat time series data as observations from a control path, parameterise a CDE's vector field using a neural network, and use the solution path as a continuously evolving hidden state. As their formulation makes them robust to irregular sampling rates, NCDEs are a powerful approach for modelling real-world data. Building on neural rough differential equations (NRDEs), we introduce Log-NCDEs, a novel, effective, and efficient method for training NCDEs. The core component of Log-NCDEs is the Log-ODE method, a tool from the study of rough paths for approximating a CDE's solution. Log-NCDEs are shown to outperform NCDEs, NRDEs, the linear recurrent unit, S5, and MAMBA on a range of multivariate time series datasets with up to $50{,}000$ observations.