Fading memory echo state networks are universal
This provides a theoretical foundation for ESNs in time-series modeling, but it is incremental as it extends prior universality results to include specific properties.
The paper tackled the problem of constructing universal approximants for input/output systems using echo state networks (ESNs) under uniform boundedness conditions, and showed that a family of ESNs with echo state and fading memory properties can achieve this universality.
Echo state networks (ESNs) have been recently proved to be universal approximants for input/output systems with respect to various $L ^p$-type criteria. When $1\leq p< \infty$, only $p$-integrability hypotheses need to be imposed, while in the case $p=\infty$ a uniform boundedness hypotheses on the inputs is required. This note shows that, in the last case, a universal family of ESNs can be constructed that contains exclusively elements that have the echo state and the fading memory properties. This conclusion could not be drawn with the results and methods available so far in the literature.