Model-Size Reduction for Reservoir Computing by Concatenating Internal States Through Time
This work addresses the need for more energy-efficient RC-based systems, such as FPGAs and photonic systems, by providing a method to reduce model size, though it is incremental as it builds on existing RC frameworks.
The study tackled the problem of reducing computational resources for reservoir computing (RC) in edge computing by proposing methods that concatenate past or drifting reservoir states to the output layer, enabling a reservoir size reduction of up to one tenth without significantly increasing regression error on time-series prediction tasks like the generalized Henon-map and NARMA.
Reservoir computing (RC) is a machine learning algorithm that can learn complex time series from data very rapidly based on the use of high-dimensional dynamical systems, such as random networks of neurons, called "reservoirs." To implement RC in edge computing, it is highly important to reduce the amount of computational resources that RC requires. In this study, we propose methods that reduce the size of the reservoir by inputting the past or drifting states of the reservoir to the output layer at the current time step. These proposed methods are analyzed based on information processing capacity, which is a performance measure of RC proposed by Dambre et al. (2012). In addition, we evaluate the effectiveness of the proposed methods on time-series prediction tasks: the generalized Henon-map and NARMA. On these tasks, we found that the proposed methods were able to reduce the size of the reservoir up to one tenth without a substantial increase in regression error. Because the applications of the proposed methods are not limited to a specific network structure of the reservoir, the proposed methods could further improve the energy efficiency of RC-based systems, such as FPGAs and photonic systems.