Reservoir Computing Universality With Stochastic Inputs
This establishes universality for widely used reservoir systems, addressing scalability in supervised machine learning for high-dimensional data, though it is incremental as it extends existing theory to stochastic inputs.
The paper proves that three families of reservoir computers, including linear reservoir systems with polynomial/neural network readouts and systems with linear readouts like trigonometric state-affine systems and echo state networks, are universal approximators for stochastic discrete-time inputs under L^p criteria, enabling high-dimensional applications without requiring bounded inputs or fading memory.
The universal approximation properties with respect to $L ^p $-type criteria of three important families of reservoir computers with stochastic discrete-time semi-infinite inputs is shown. First, it is proved that linear reservoir systems with either polynomial or neural network readout maps are universal. More importantly, it is proved that the same property holds for two families with linear readouts, namely, trigonometric state-affine systems and echo state networks, which are the most widely used reservoir systems in applications. The linearity in the readouts is a key feature in supervised machine learning applications. It guarantees that these systems can be used in high-dimensional situations and in the presence of large datasets. The $L ^p $ criteria used in this paper allow the formulation of universality results that do not necessarily impose almost sure uniform boundedness in the inputs or the fading memory property in the filter that needs to be approximated.