Towards a Calculus of Echo State Networks
This provides a foundational theoretical tool for researchers in reservoir computing, enabling precise performance predictions, though it is incremental as it builds on prior upper-bound analyses.
The authors tackled the problem of analytically characterizing memory in echo state networks, a class of reservoir computing, by developing a framework that calculates the entire memory curve based on system structure and input properties, validating it with numerical simulations across various system sizes and spectral radii.
Reservoir computing is a recent trend in neural networks which uses the dynamical perturbations on the phase space of a system to compute a desired target function. We present how one can formulate an expectation of system performance in a simple class of reservoir computing called echo state networks. In contrast with previous theoretical frameworks, which only reveal an upper bound on the total memory in the system, we analytically calculate the entire memory curve as a function of the structure of the system and the properties of the input and the target function. We demonstrate the precision of our framework by validating its result for a wide range of system sizes and spectral radii. Our analytical calculation agrees with numerical simulations. To the best of our knowledge this work presents the first exact analytical characterization of the memory curve in echo state networks.