A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks
This work addresses the training inefficiency in neuromorphic computing for applications like vision processing, though it appears incremental as it builds on existing SNN-ANN hybrid approaches.
The authors tackled the challenge of training deep spiking neural networks (SNNs) due to non-differentiable spiking functions by proposing a tandem learning framework with an auxiliary ANN for error back-propagation, achieving competitive accuracy and at least an order of magnitude reduction in inference time and synaptic operations compared to state-of-the-art SNNs.
Spiking neural networks (SNNs) represent the most prominent biologically inspired computing model for neuromorphic computing (NC) architectures. However, due to the non-differentiable nature of spiking neuronal functions, the standard error back-propagation algorithm is not directly applicable to SNNs. In this work, we propose a tandem learning framework, that consists of an SNN and an Artificial Neural Network (ANN) coupled through weight sharing. The ANN is an auxiliary structure that facilitates the error back-propagation for the training of the SNN at the spike-train level. To this end, we consider the spike count as the discrete neural representation in the SNN, and design ANN neuronal activation function that can effectively approximate the spike count of the coupled SNN. The proposed tandem learning rule demonstrates competitive pattern recognition and regression capabilities on both the conventional frame-based and event-based vision datasets, with at least an order of magnitude reduced inference time and total synaptic operations over other state-of-the-art SNN implementations. Therefore, the proposed tandem learning rule offers a novel solution to training efficient, low latency, and high accuracy deep SNNs with low computing resources.