ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks
This work addresses accuracy limitations in SNNs for energy-efficient AI applications, representing an incremental improvement with specific gains.
The paper tackles the problem of reduced accuracy in Spiking Neural Networks (SNNs) due to binary spike activations by proposing ReverB-SNN, which uses real-valued activations and binary weights, and it demonstrates consistent outperformance over state-of-the-art methods in experiments across various architectures and datasets.
The Spiking Neural Network (SNN), a biologically inspired neural network infrastructure, has garnered significant attention recently. SNNs utilize binary spike activations for efficient information transmission, replacing multiplications with additions, thereby enhancing energy efficiency. However, binary spike activation maps often fail to capture sufficient data information, resulting in reduced accuracy. To address this challenge, we advocate reversing the bit of the weight and activation for SNNs, called \textbf{ReverB-SNN}, inspired by recent findings that highlight greater accuracy degradation from quantizing activations compared to weights. Specifically, our method employs real-valued spike activations alongside binary weights in SNNs. This preserves the event-driven and multiplication-free advantages of standard SNNs while enhancing the information capacity of activations. Additionally, we introduce a trainable factor within binary weights to adaptively learn suitable weight amplitudes during training, thereby increasing network capacity. To maintain efficiency akin to vanilla \textbf{ReverB-SNN}, our trainable binary weight SNNs are converted back to standard form using a re-parameterization technique during inference. Extensive experiments across various network architectures and datasets, both static and dynamic, demonstrate that our approach consistently outperforms state-of-the-art methods.