Mean Field Theory of Dynamical Systems Driven by External Signals
This work addresses the theoretical understanding of echo state networks for researchers in dynamical systems and machine learning, but it is incremental as it applies existing mean field theory to a specific model.
The paper tackles the problem of modeling echo state networks, a type of neural network used in machine learning, by developing a mean field theory that reduces network dynamics to a single collective variable, predicting behaviors like steady states under multiple signals and nonstationary distributions under a single signal, with results validated against numerical simulations including the largest Lyapunov exponent.
Dynamical systems driven by strong external signals are ubiquituous in nature and engineering. Here we study "echo state networks", networks of a large number of randomly connected nodes, which represent a simple model of a neural network, and have important applications in machine learning. We develop a mean field theory of echo state networks. The dynamics of the network is captured by the evolution law, similar to a logistic map, for a single collective variable. When the network is driven by many independent external signals, this collective variable reaches a steady state. But when the network is driven by a single external signal, the collective variable is nonstationnary but can be characterised by its time averaged distribution. The predictions of the mean field theory, including the value of the largest Lyaponuov exponent, are compared with the numerical integration of the equations of motion.