Three factor delay learning rules for spiking neural networks
This work addresses the challenge of real-time operation in resource-constrained neuromorphic processors by enabling on-device learning with reduced memory requirements, though it is incremental as it builds on existing delay methods.
The paper tackles the problem of limited learnable parameters in Spiking Neural Networks (SNNs) for temporal pattern recognition by introducing synaptic and axonal delays with online three-factor learning rules, resulting in up to 20% higher accuracy over weights-only baselines, 6.6x smaller model size, and 67% lower inference latency with only a 2.4% accuracy drop.
Spiking Neural Networks (SNNs) are dynamical systems that operate on spatiotemporal data, yet their learnable parameters are often limited to synaptic weights, contributing little to temporal pattern recognition. Learnable parameters that delay spike times can improve classification performance in temporal tasks, but existing methods rely on large networks and offline learning, making them unsuitable for real-time operation in resource-constrained environments. In this paper, we introduce synaptic and axonal delays to leaky integrate and fire (LIF)-based feedforward and recurrent SNNs, and propose three-factor learning rules to simultaneously learn delay parameters online. We employ a smooth Gaussian surrogate to approximate spike derivatives exclusively for the eligibility trace calculation, and together with a top-down error signal determine parameter updates. Our experiments show that incorporating delays improves accuracy by up to 20% over a weights-only baseline, and for networks with similar parameter counts, jointly learning weights and delays yields up to 14% higher accuracy. On the SHD speech recognition dataset, our method achieves similar accuracy to offline backpropagation-based approaches. Compared to state-of-the-art methods, it reduces model size by 6.6x and inference latency by 67%, with only a 2.4% drop in classification accuracy. Our findings benefit the design of power and area-constrained neuromorphic processors by enabling on-device learning and lowering memory requirements.