On Dominant Manifolds in Reservoir Computing Networks
This work provides insights into the geometric properties of trained recurrent networks, which is incremental for researchers in time-series modeling and dynamical systems.
The paper tackled the problem of understanding how training shapes the geometry of recurrent network dynamics in Reservoir Computing networks for temporal forecasting, showing that the dimensionality and structure of dominant manifolds are linked to the intrinsic dimensionality and information content of training data, with simulations illustrating eigenvalue motion during training.
Understanding how training shapes the geometry of recurrent network dynamics is a central problem in time-series modeling. We study the emergence of low-dimensional dominant manifolds in the training of Reservoir Computing (RC) networks for temporal forecasting tasks. For a simplified linear and continuous-time reservoir model, we link the dimensionality and structure of the dominant modes directly to the intrinsic dimensionality and information content of the training data. In particular, for training data generated by an autonomous dynamical system, we relate the dominant modes of the trained reservoir to approximations of the Koopman eigenfunctions of the original system, illuminating an explicit connection between reservoir computing and the Dynamic Mode Decomposition algorithm. We illustrate the eigenvalue motion that generates the dominant manifolds during training in simulation, and discuss generalization to nonlinear RC via tangent dynamics and differential p-dominance.