Improving Surrogate Gradient Learning in Spiking Neural Networks via Regularization and Normalization
This work addresses the accuracy gap in SNNs for potential low-power AI applications, but it appears incremental as it applies existing techniques without claiming major breakthroughs.
The authors tackled the problem of lower accuracy in spiking neural networks (SNNs) compared to analog networks by applying regularization and normalization techniques to improve surrogate gradient learning, but no concrete results or numbers are provided in the abstract.
Spiking neural networks (SNNs) are different from the classical networks used in deep learning: the neurons communicate using electrical impulses called spikes, just like biological neurons. SNNs are appealing for AI technology, because they could be implemented on low power neuromorphic chips. However, SNNs generally remain less accurate than their analog counterparts. In this report, we examine various regularization and normalization techniques with the goal of improving surrogate gradient learning in SNNs.