Noisy Softplus: an activation function that enables SNNs to be trained as ANNs
This research provides an effective training method for SNNs, benefiting researchers in neuromorphic computing by addressing a key bottleneck in achieving biological characteristics and performance parity with ANNs, though it appears incremental as an extension of prior work.
The paper tackles the challenge of training spiking neural networks (SNNs) by introducing Noisy Softplus, an activation function that allows SNNs to be trained using traditional algorithms like Back Propagation, enabling direct weight transfer to spiking versions without conversion. This approach increased classification accuracy and helped close the performance gap between SNNs and ANNs, though no specific numerical results are provided.
We extended the work of proposed activation function, Noisy Softplus, to fit into training of layered up spiking neural networks (SNNs). Thus, any ANN employing Noisy Softplus neurons, even of deep architecture, can be trained simply by the traditional algorithm, for example Back Propagation (BP), and the trained weights can be directly used in the spiking version of the same network without any conversion. Furthermore, the training method can be generalised to other activation units, for instance Rectified Linear Units (ReLU), to train deep SNNs off-line. This research is crucial to provide an effective approach for SNN training, and to increase the classification accuracy of SNNs with biological characteristics and to close the gap between the performance of SNNs and ANNs.