RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networks
This addresses the issue of catastrophic information loss due to quantization in SNNs, which is crucial for energy-efficient hardware implementations, though it appears incremental as it builds on existing SNN methods.
The paper tackles the quantization error problem in Spiking Neural Networks (SNNs) by proposing a regularizing membrane potential loss (RMP-Loss) to adjust the distribution, resulting in consistent outperformance over previous state-of-the-art methods across various network architectures and datasets.
Spiking Neural Networks (SNNs) as one of the biology-inspired models have received much attention recently. It can significantly reduce energy consumption since they quantize the real-valued membrane potentials to 0/1 spikes to transmit information thus the multiplications of activations and weights can be replaced by additions when implemented on hardware. However, this quantization mechanism will inevitably introduce quantization error, thus causing catastrophic information loss. To address the quantization error problem, we propose a regularizing membrane potential loss (RMP-Loss) to adjust the distribution which is directly related to quantization error to a range close to the spikes. Our method is extremely simple to implement and straightforward to train an SNN. Furthermore, it is shown to consistently outperform previous state-of-the-art methods over different network architectures and datasets.