Deep Time-Delay Reservoir Computing: Dynamics and Memory Capacity
This work addresses memory optimization in reservoir computing for machine learning applications, presenting incremental improvements through novel dynamical analysis.
The paper tackles the problem of optimizing memory capacity in deep time-delay reservoir computing by analyzing dynamical properties like bifurcations and conditional Lyapunov exponents, showing that resonances between clock cycles and delays can boost specific memory capacity degrees, with configurations achieving either high nonlinear or long linear memory capacity.
The Deep Time-Delay Reservoir Computing concept utilizes unidirectionally connected systems with time-delays for supervised learning. We present how the dynamical properties of a deep Ikeda-based reservoir are related to its memory capacity (MC) and how that can be used for optimization. In particular, we analyze bifurcations of the corresponding autonomous system and compute conditional Lyapunov exponents, which measure the generalized synchronization between the input and the layer dynamics. We show how the MC is related to the systems distance to bifurcations or magnitude of the conditional Lyapunov exponent. The interplay of different dynamical regimes leads to a adjustable distribution between linear and nonlinear MC. Furthermore, numerical simulations show resonances between clock cycle and delays of the layers in all degrees of the MC. Contrary to MC losses in a single-layer reservoirs, these resonances can boost separate degrees of the MC and can be used, e.g., to design a system with maximum linear MC. Accordingly, we present two configurations that empower either high nonlinear MC or long time linear MC.