Minimal Deterministic Echo State Networks Outperform Random Reservoirs in Learning Chaotic Dynamics
This work addresses the problem of unreliable performance in ESNs for researchers and practitioners modeling chaotic dynamics, offering a more robust and efficient alternative.
The authors tackled the sensitivity of echo state networks (ESNs) to hyperparameters and random initialization in modeling chaotic systems, demonstrating that minimal deterministic ESNs (MESNs) achieve up to a 41% error reduction compared to standard ESNs.
Machine learning (ML) is widely used to model chaotic systems. Among ML approaches, echo state networks (ESNs) have received considerable attention due to their simple construction and fast training. However, ESN performance is highly sensitive to hyperparameter choices and to its random initialization. In this work, we demonstrate that ESNs constructed using deterministic rules and simple topologies (MESNs) outperform standard ESNs in the task of chaotic attractor reconstruction. We use a dataset of more than 90 chaotic systems to benchmark 10 different minimal deterministic reservoir initializations. We find that MESNs obtain up to a 41% reduction in error compared to standard ESNs. Furthermore, we show that the MESNs are more robust, exhibiting less inter-run variation, and have the ability to reuse hyperparameters across different systems. Our results illustrate how structured simplicity in ESN design can outperform stochastic complexity in learning chaotic dynamics.