Integer Echo State Networks: Efficient Reservoir Computing for Digital Hardware
This work addresses the problem of implementing reservoir computing efficiently on digital hardware for applications like time-series classification, though it is incremental as it builds on existing Echo State Networks.
The authors tackled the inefficiency of Echo State Networks on digital hardware by proposing an integer-based approximation that uses hyperdimensional computing principles, resulting in dramatic improvements in memory footprint and computational efficiency with minimal performance loss, as confirmed by experiments on a field-programmable gate array showing much higher energy efficiency.
We propose an approximation of Echo State Networks (ESN) that can be efficiently implemented on digital hardware based on the mathematics of hyperdimensional computing. The reservoir of the proposed integer Echo State Network (intESN) is a vector containing only n-bits integers (where n<8 is normally sufficient for a satisfactory performance). The recurrent matrix multiplication is replaced with an efficient cyclic shift operation. The proposed intESN approach is verified with typical tasks in reservoir computing: memorizing of a sequence of inputs; classifying time-series; learning dynamic processes. Such architecture results in dramatic improvements in memory footprint and computational efficiency, with minimal performance loss. The experiments on a field-programmable gate array confirm that the proposed intESN approach is much more energy efficient than the conventional ESN.