S-TLLR: STDP-inspired Temporal Local Learning Rule for Spiking Neural Networks
This work addresses the problem of high computational cost in SNN training for researchers and practitioners in neuromorphic computing, offering a more efficient method suitable for low-power edge applications, though it is incremental as it builds on existing STDP-inspired approaches.
The paper tackled the challenge of training Spiking Neural Networks (SNNs) efficiently for edge devices by proposing S-TLLR, a temporal local learning rule inspired by STDP, which achieved comparable accuracy to back-propagation through time while reducing memory usage by 5-50 times and MAC operations by 1.3-6.6 times across various event-based datasets.
Spiking Neural Networks (SNNs) are biologically plausible models that have been identified as potentially apt for deploying energy-efficient intelligence at the edge, particularly for sequential learning tasks. However, training of SNNs poses significant challenges due to the necessity for precise temporal and spatial credit assignment. Back-propagation through time (BPTT) algorithm, whilst the most widely used method for addressing these issues, incurs high computational cost due to its temporal dependency. In this work, we propose S-TLLR, a novel three-factor temporal local learning rule inspired by the Spike-Timing Dependent Plasticity (STDP) mechanism, aimed at training deep SNNs on event-based learning tasks. Furthermore, S-TLLR is designed to have low memory and time complexities, which are independent of the number of time steps, rendering it suitable for online learning on low-power edge devices. To demonstrate the scalability of our proposed method, we have conducted extensive evaluations on event-based datasets spanning a wide range of applications, such as image and gesture recognition, audio classification, and optical flow estimation. In all the experiments, S-TLLR achieved high accuracy, comparable to BPTT, with a reduction in memory between $5-50\times$ and multiply-accumulate (MAC) operations between $1.3-6.6\times$.