A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration
This work addresses the efficiency and accuracy challenges in SNN conversion for applications in neuromorphic computing, representing an incremental improvement over existing conversion methods.
The paper tackles the problem of converting pre-trained artificial neural networks (ANNs) to spiking neural networks (SNNs) by proposing a calibration method that corrects conversion errors layer-by-layer, resulting in up to a 69% increase in top-1 accuracy for MobileNet on ImageNet compared to baselines.
Spiking Neural Network (SNN) has been recognized as one of the next generation of neural networks. Conventionally, SNN can be converted from a pre-trained ANN by only replacing the ReLU activation to spike activation while keeping the parameters intact. Perhaps surprisingly, in this work we show that a proper way to calibrate the parameters during the conversion of ANN to SNN can bring significant improvements. We introduce SNN Calibration, a cheap but extraordinarily effective method by leveraging the knowledge within a pre-trained Artificial Neural Network (ANN). Starting by analyzing the conversion error and its propagation through layers theoretically, we propose the calibration algorithm that can correct the error layer-by-layer. The calibration only takes a handful number of training data and several minutes to finish. Moreover, our calibration algorithm can produce SNN with state-of-the-art architecture on the large-scale ImageNet dataset, including MobileNet and RegNet. Extensive experiments demonstrate the effectiveness and efficiency of our algorithm. For example, our advanced pipeline can increase up to 69% top-1 accuracy when converting MobileNet on ImageNet compared to baselines. Codes are released at https://github.com/yhhhli/SNN_Calibration.