Short note on the behavior of recurrent neural network for noisy dynamical system
This work addresses the robustness and adaptability of LSTMs in simulating noisy dynamical systems, which is incremental as it builds on existing LSTM applications.
The study examined how LSTM networks behave when trained on noisy Mackey-Glass time series, finding that higher training noise leads to reliance on autonomous dynamics, reducing susceptibility to perturbations but increasing relaxation times, while noiseless training makes LSTMs sensitive to small perturbations but adaptable to input changes.
The behavior of recurrent neural network for the data-driven simulation of noisy dynamical systems is studied by training a set of Long Short-Term Memory Networks (LSTM) on the Mackey-Glass time series with a wide range of noise level. It is found that, as the training noise becomes larger, LSTM learns to depend more on its autonomous dynamics than the noisy input data. As a result, LSTM trained on noisy data becomes less susceptible to the perturbation in the data, but has a longer relaxation timescale. On the other hand, when trained on noiseless data, LSTM becomes extremely sensitive to a small perturbation, but is able to adjusts to the changes in the input data.