Neuroevolution on the Edge of Chaos
This work addresses performance optimization in recurrent neural networks for researchers, offering a more efficient alternative to computationally intensive neuroevolution methods.
The paper tackled the problem of optimizing computational performance in recurrent neural networks by confirming that echo state networks perform best at the edge of chaos and showing that locally connected echo state networks combine simplicity with high performance, outperforming evolved networks while being more efficient.
Echo state networks represent a special type of recurrent neural networks. Recent papers stated that the echo state networks maximize their computational performance on the transition between order and chaos, the so-called edge of chaos. This work confirms this statement in a comprehensive set of experiments. Furthermore, the echo state networks are compared to networks evolved via neuroevolution. The evolved networks outperform the echo state networks, however, the evolution consumes significant computational resources. It is demonstrated that echo state networks with local connections combine the best of both worlds, the simplicity of random echo state networks and the performance of evolved networks. Finally, it is shown that evolution tends to stay close to the ordered side of the edge of chaos.