Sequential Learning from Noisy Data: Data-Assimilation Meets Echo-State Network
This addresses the challenge of noisy input training for dynamic predictors, which is incremental as it adapts existing methods to improve performance in specific scenarios.
The paper tackles training recurrent neural networks from noisy data by developing a sequential algorithm that combines an echo-state network with an ensemble Kalman filter, resulting in the KalT-ESN method, which outperforms traditional least square training on noisy synthetic and real-world datasets while remaining computationally efficient.
This paper explores the problem of training a recurrent neural network from noisy data. While neural network based dynamic predictors perform well with noise-free training data, prediction with noisy inputs during training phase poses a significant challenge. Here a sequential training algorithm is developed for an echo-state network (ESN) by incorporating noisy observations using an ensemble Kalman filter. The resultant Kalman-trained echo-state network (KalT-ESN) outperforms the traditionally trained ESN with least square algorithm while still being computationally cheap. The proposed method is demonstrated on noisy observations from three systems: two synthetic datasets from chaotic dynamical systems and a set of real-time traffic data.