Forward Direct Feedback Alignment for Online Gradient Estimates of Spiking Neural Networks
This work addresses the need for energy-efficient training algorithms for SNNs on neuromorphic hardware, representing an incremental improvement over existing methods.
The authors tackled the problem of training spiking neural networks (SNNs) on neuromorphic hardware where backpropagation is infeasible, by proposing the Spiking Forward Direct Feedback Alignment (SFDFA) algorithm, which achieves higher performance and convergence rates compared to competitor algorithms.
There is an interest in finding energy efficient alternatives to current state of the art neural network training algorithms. Spiking neural network are a promising approach, because they can be simulated energy efficiently on neuromorphic hardware platforms. However, these platforms come with limitations on the design of the training algorithm. Most importantly, backpropagation cannot be implemented on those. We propose a novel neuromorphic algorithm, the \textit{Spiking Forward Direct Feedback Alignment} (SFDFA) algorithm, an adaption of \textit{Forward Direct Feedback Alignment} to train SNNs. SFDFA estimates the weights between output and hidden neurons as feedback connections. The main contribution of this paper is to describe how exact local gradients of spikes can be computed in an online manner while taking into account the intra-neuron dependencies between post-synaptic spikes and derive a dynamical system for neuromorphic hardware compatibility. We compare the SFDFA algorithm with a number of competitor algorithms and show that the proposed algorithm achieves higher performance and convergence rates.