Heterogeneous Neuronal and Synaptic Dynamics for Spike-Efficient Unsupervised Learning: Theory and Design Principles
This addresses the challenge of spike-efficient unsupervised learning in neuromorphic computing, offering incremental improvements in energy efficiency and performance for time series tasks.
The paper tackles the problem of high spiking activity in Recurrent Spiking Neural Networks (RSNNs) by introducing heterogeneity in neuronal and synaptic dynamics, which reduces spiking activity while improving prediction performance and memory capacity. Empirical results show that optimized heterogeneous RSNNs increase performance and reduce spiking activity compared to homogeneous RSNNs.
This paper shows that the heterogeneity in neuronal and synaptic dynamics reduces the spiking activity of a Recurrent Spiking Neural Network (RSNN) while improving prediction performance, enabling spike-efficient (unsupervised) learning. We analytically show that the diversity in neurons' integration/relaxation dynamics improves an RSNN's ability to learn more distinct input patterns (higher memory capacity), leading to improved classification and prediction performance. We further prove that heterogeneous Spike-Timing-Dependent-Plasticity (STDP) dynamics of synapses reduce spiking activity but preserve memory capacity. The analytical results motivate Heterogeneous RSNN design using Bayesian optimization to determine heterogeneity in neurons and synapses to improve $\mathcal{E}$, defined as the ratio of spiking activity and memory capacity. The empirical results on time series classification and prediction tasks show that optimized HRSNN increases performance and reduces spiking activity compared to a homogeneous RSNN.