Takens-inspired neuromorphic processor: a downsizing tool for random recurrent neural networks via feature extraction
This work addresses the challenge of downsizing neural networks for time series prediction, offering a domain-specific solution that is incremental in nature.
The paper tackles the problem of reducing neuron count in random recurrent neural networks for time series prediction by merging Takens-based attractor reconstruction with machine learning for feature extraction, resulting in a 15-fold reduction in network size while stabilizing an arrhythmic neural model.
We describe a new technique which minimizes the amount of neurons in the hidden layer of a random recurrent neural network (rRNN) for time series prediction. Merging Takens-based attractor reconstruction methods with machine learning, we identify a mechanism for feature extraction that can be leveraged to lower the network size. We obtain criteria specific to the particular prediction task and derive the scaling law of the prediction error. The consequences of our theory are demonstrated by designing a Takens-inspired hybrid processor, which extends a rRNN with a priori designed delay external memory. Our hybrid architecture is therefore designed including both, real and virtual nodes. Via this symbiosis, we show performance of the hybrid processor by stabilizing an arrhythmic neural model. Thanks to our obtained design rules, we can reduce the stabilizing neural network's size by a factor of 15 with respect to a standard system.