Estimating Vector Fields from Noisy Time Series
This work significantly improves the robustness of differential equation learning methods to noise, which is a critical problem for researchers and practitioners working with real-world, noisy time series data.
This paper addresses the challenge of estimating vector fields from highly noisy time series data, where existing methods struggle with noise levels as low as 1%. The authors propose a neural network architecture combined with an alternating minimization scheme and multiple trajectories, which enables learning accurate vector fields from data with 10% Gaussian noise.
While there has been a surge of recent interest in learning differential equation models from time series, methods in this area typically cannot cope with highly noisy data. We break this problem into two parts: (i) approximating the unknown vector field (or right-hand side) of the differential equation, and (ii) dealing with noise. To deal with (i), we describe a neural network architecture consisting of tensor products of one-dimensional neural shape functions. For (ii), we propose an alternating minimization scheme that switches between vector field training and filtering steps, together with multiple trajectories of training data. We find that the neural shape function architecture retains the approximation properties of dense neural networks, enables effective computation of vector field error, and allows for graphical interpretability, all for data/systems in any finite dimension $d$. We also study the combination of either our neural shape function method or existing differential equation learning methods with alternating minimization and multiple trajectories. We find that retrofitting any learning method in this way boosts the method's robustness to noise. While in their raw form the methods struggle with 1% Gaussian noise, after retrofitting, they learn accurate vector fields from data with 10% Gaussian noise.