Deep Echo State Network (DeepESN): A Brief Survey
This is an incremental survey summarizing existing advancements in DeepESNs for researchers in neural networks.
The paper surveys the Deep Echo State Network (DeepESN) model, which provides an efficient approach for designing deep neural networks for temporal data and explores the properties of hierarchical recurrent layers in RNNs.
The study of deep recurrent neural networks (RNNs) and, in particular, of deep Reservoir Computing (RC) is gaining an increasing research attention in the neural networks community. The recently introduced Deep Echo State Network (DeepESN) model opened the way to an extremely efficient approach for designing deep neural networks for temporal data. At the same time, the study of DeepESNs allowed to shed light on the intrinsic properties of state dynamics developed by hierarchical compositions of recurrent layers, i.e. on the bias of depth in RNNs architectural design. In this paper, we summarize the advancements in the development, analysis and applications of DeepESNs.