Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
This work addresses the problem of enabling efficient, biologically plausible learning in spiking neural networks for researchers and engineers in neuromorphic computing, though it is incremental as it builds on existing synthetic gradient methods.
The authors tackled the challenge of aligning synaptic plasticity dynamics in spiking neural networks with gradient backpropagation requirements by developing Deep Continuous Local Learning (DECOLLE), which enables deep spatio-temporal learning from spikes using local information and achieves performance comparable to state-of-the-art on MNIST and DvsGesture datasets.
A growing body of work underlines striking similarities between biological neural networks and recurrent, binary neural networks. A relatively smaller body of work, however, discusses similarities between learning dynamics employed in deep artificial neural networks and synaptic plasticity in spiking neural networks. The challenge preventing this is largely caused by the discrepancy between the dynamical properties of synaptic plasticity and the requirements for gradient backpropagation. Learning algorithms that approximate gradient backpropagation using locally synthesized gradients can overcome this challenge. Here, we show that synthetic gradients enable the derivation of Deep Continuous Local Learning (DECOLLE) in spiking neural networks. DECOLLE is capable of learning deep spatio-temporal representations from spikes relying solely on local information. Synaptic plasticity rules are derived systematically from user-defined cost functions and neural dynamics by leveraging existing autodifferentiation methods of machine learning frameworks. We benchmark our approach on the MNIST and the event-based neuromorphic DvsGesture dataset, on which DECOLLE performs comparably to the state-of-the-art. DECOLLE networks provide continuously learning machines that are relevant to biology and supportive of event-based, low-power computer vision architectures matching the accuracies of conventional computers on tasks where temporal precision and speed are essential.