An Experimental Analysis of the Echo State Network Initialization Using the Particle Swarm Optimization
This work addresses performance optimization for ESNs in supervised learning, but it is incremental as it combines existing methods (ESN and PSO) without introducing a fundamentally new approach.
The paper tackles the problem of initializing Echo State Networks (ESNs) by proposing a hybrid method that uses Particle Swarm Optimization (PSO) to set initial hidden-hidden weights, and it shows empirical improvements over canonical ESNs on benchmark tasks.
This article introduces a robust hybrid method for solving supervised learning tasks, which uses the Echo State Network (ESN) model and the Particle Swarm Optimization (PSO) algorithm. An ESN is a Recurrent Neural Network with the hidden-hidden weights fixed in the learning process. The recurrent part of the network stores the input information in internal states of the network. Another structure forms a free-memory method used as supervised learning tool. The setting procedure for initializing the recurrent structure of the ESN model can impact on the model performance. On the other hand, the PSO has been shown to be a successful technique for finding optimal points in complex spaces. Here, we present an approach to use the PSO for finding some initial hidden-hidden weights of the ESN model. We present empirical results that compare the canonical ESN model with this hybrid method on a wide range of benchmark problems.