Reservoir Topology in Deep Echo State Networks
This work addresses performance enhancement in reservoir computing for machine learning practitioners, but it appears incremental as it builds on existing DeepESN methods with specific topological modifications.
The paper tackled the problem of improving predictive performance in Deep Echo State Networks by studying constrained reservoir topologies, showing that combining deep reservoirs with structured recurrent units, especially using permutation matrices, leads to significant gains.
Deep Echo State Networks (DeepESNs) recently extended the applicability of Reservoir Computing (RC) methods towards the field of deep learning. In this paper we study the impact of constrained reservoir topologies in the architectural design of deep reservoirs, through numerical experiments on several RC benchmarks. The major outcome of our investigation is to show the remarkable effect, in terms of predictive performance gain, achieved by the synergy between a deep reservoir construction and a structured organization of the recurrent units in each layer. Our results also indicate that a particularly advantageous architectural setting is obtained in correspondence of DeepESNs where reservoir units are structured according to a permutation recurrent matrix.