FTBC: Forward Temporal Bias Correction for Optimizing ANN-SNN Conversion
This work addresses the problem of enhancing energy-efficient SNN performance for AI applications, but it is incremental as it builds on existing ANN-SNN conversion methods.
The paper tackles the challenge of improving ANN-SNN conversion accuracy for energy-efficient computing by introducing a Forward Temporal Bias Correction (FTBC) technique, which achieves notable accuracy increases on CIFAR-10/100 and ImageNet datasets without computational overhead.
Spiking Neural Networks (SNNs) offer a promising avenue for energy-efficient computing compared with Artificial Neural Networks (ANNs), closely mirroring biological neural processes. However, this potential comes with inherent challenges in directly training SNNs through spatio-temporal backpropagation -- stemming from the temporal dynamics of spiking neurons and their discrete signal processing -- which necessitates alternative ways of training, most notably through ANN-SNN conversion. In this work, we introduce a lightweight Forward Temporal Bias Correction (FTBC) technique, aimed at enhancing conversion accuracy without the computational overhead. We ground our method on provided theoretical findings that through proper temporal bias calibration the expected error of ANN-SNN conversion can be reduced to be zero after each time step. We further propose a heuristic algorithm for finding the temporal bias only in the forward pass, thus eliminating the computational burden of backpropagation and we evaluate our method on CIFAR-10/100 and ImageNet datasets, achieving a notable increase in accuracy on all datasets. Codes are released at a GitHub repository.