Echo state networks are universal
This foundational result provides theoretical guarantees for reservoir computing in machine learning, addressing the approximation capabilities of echo state networks for time-series tasks.
The paper proves that echo state networks can universally approximate any discrete-time fading memory filter with bounded inputs over infinite time intervals, establishing them as a simple finite-dimensional neural network model with a linear readout.
This paper shows that echo state networks are universal uniform approximants in the context of discrete-time fading memory filters with uniformly bounded inputs defined on negative infinite times. This result guarantees that any fading memory input/output system in discrete time can be realized as a simple finite-dimensional neural network-type state-space model with a static linear readout map. This approximation is valid for infinite time intervals. The proof of this statement is based on fundamental results, also presented in this work, about the topological nature of the fading memory property and about reservoir computing systems generated by continuous reservoir maps.