InfLoR-SNN: Reducing Information Loss for Spiking Neural Networks
This work addresses efficiency and accuracy issues in SNNs, which are important for low-energy AI applications, but it is incremental as it builds on existing SNN training methods.
The paper tackled information loss in Spiking Neural Networks (SNNs) caused by the 'Hard Reset' mechanism and quantization errors, proposing a 'Soft Reset' mechanism and Membrane Potential Rectifier (MPR) to improve performance, with results showing that these methods outperform vanilla SNNs on static and dynamic datasets.
The Spiking Neural Network (SNN) has attracted more and more attention recently. It adopts binary spike signals to transmit information. Benefitting from the information passing paradigm of SNNs, the multiplications of activations and weights can be replaced by additions, which are more energy-efficient. However, its "Hard Reset" mechanism for the firing activity would ignore the difference among membrane potentials when the membrane potential is above the firing threshold, causing information loss. Meanwhile, quantifying the membrane potential to 0/1 spikes at the firing instants will inevitably introduce the quantization error thus bringing about information loss too. To address these problems, we propose to use the "Soft Reset" mechanism for the supervised training-based SNNs, which will drive the membrane potential to a dynamic reset potential according to its magnitude, and Membrane Potential Rectifier (MPR) to reduce the quantization error via redistributing the membrane potential to a range close to the spikes. Results show that the SNNs with the "Soft Reset" mechanism and MPR outperform their vanilla counterparts on both static and dynamic datasets.