Prediction of Dynamical time Series Using Kernel Based Regression and Smooth Splines
This work addresses the challenge of accurate prediction in noisy dynamical systems for fields like signal processing or physics, though it appears incremental as it builds on existing kernel methods by adding denoising.
The paper tackles the problem of predicting dynamical time series with additive noise by separating denoising and prediction, showing that combining smooth splines with kernel-based regression yields predictors that are more accurate, typically by a factor of 2 or more, and in some cases up to 100 times better.
Prediction of dynamical time series with additive noise using support vector machines or kernel based regression has been proved to be consistent for certain classes of discrete dynamical systems. Consistency implies that these methods are effective at computing the expected value of a point at a future time given the present coordinates. However, the present coordinates themselves are noisy, and therefore, these methods are not necessarily effective at removing noise. In this article, we consider denoising and prediction as separate problems for flows, as opposed to discrete time dynamical systems, and show that the use of smooth splines is more effective at removing noise. Combination of smooth splines and kernel based regression yields predictors that are more accurate on benchmarks typically by a factor of 2 or more. We prove that kernel based regression in combination with smooth splines converges to the exact predictor for time series extracted from any compact invariant set of any sufficiently smooth flow. As a consequence of convergence, one can find examples where the combination of kernel based regression with smooth splines is superior by even a factor of $100$. The predictors that we compute operate on delay coordinate data and not the full state vector, which is typically not observable.