Embedding Theory of Reservoir Computing and Reducing Reservoir Network Using Time Delays
This work addresses the unclear mechanism of RC for researchers in machine learning and physics, offering a theoretical foundation and practical reduction in computational complexity.
The authors proved that reservoir computing (RC) is a high-dimensional embedding of nonlinear dynamical systems, enabling them to unify standard and time-delayed RC into a framework and reduce network size, achieving tasks like reconstruction and prediction with as few as a single neuron.
Reservoir computing (RC), a particular form of recurrent neural network, is under explosive development due to its exceptional efficacy and high performance in reconstruction or/and prediction of complex physical systems. However, the mechanism triggering such effective applications of RC is still unclear, awaiting deep and systematic exploration. Here, combining the delayed embedding theory with the generalized embedding theory, we rigorously prove that RC is essentially a high dimensional embedding of the original input nonlinear dynamical system. Thus, using this embedding property, we unify into a universal framework the standard RC and the time-delayed RC where we novelly introduce time delays only into the network's output layer, and we further find a trade-off relation between the time delays and the number of neurons in RC. Based on this finding, we significantly reduce the network size of RC for reconstructing and predicting some representative physical systems, and, more surprisingly, only using a single neuron reservoir with time delays is sometimes sufficient for achieving those tasks.