An Exact Mapping From ReLU Networks to Spiking Neural Networks
This enables low-power AI by providing an exact mapping for energy-efficient SNNs, though it is incremental as it builds on existing conversion methods.
The authors tackled the challenge of converting deep ReLU networks to spiking neural networks (SNNs) without performance loss, achieving zero percent drop in accuracy on datasets like CIFAR10, CIFAR100, and Places365.
Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an exact mapping from a network with Rectified Linear Units (ReLUs) to an SNN that fires exactly one spike per neuron. For our constructive proof, we assume that an arbitrary multi-layer ReLU network with or without convolutional layers, batch normalization and max pooling layers was trained to high performance on some training set. Furthermore, we assume that we have access to a representative example of input data used during training and to the exact parameters (weights and biases) of the trained ReLU network. The mapping from deep ReLU networks to SNNs causes zero percent drop in accuracy on CIFAR10, CIFAR100 and the ImageNet-like data sets Places365 and PASS. More generally our work shows that an arbitrary deep ReLU network can be replaced by an energy-efficient single-spike neural network without any loss of performance.