SPICEprop: Backpropagating Errors Through Memristive Spiking Neural Networks
This work addresses efficient neuromorphic computing for edge devices by enabling direct training on hardware, though it is incremental in improving accuracy for specific datasets.
The authors tackled training fully memristive spiking neural networks by using backpropagation through time on differentiable SPICE circuit models, achieving 97.58% accuracy on MNIST and 75.26% on Fashion-MNIST, the highest among such networks.
We present a fully memristive spiking neural network (MSNN) consisting of novel memristive neurons trained using the backpropagation through time (BPTT) learning rule. Gradient descent is applied directly to the memristive integrated-and-fire (MIF) neuron designed using analog SPICE circuit models, which generates distinct depolarization, hyperpolarization, and repolarization voltage waveforms. Synaptic weights are trained by BPTT using the membrane potential of the MIF neuron model and can be processed on memristive crossbars. The natural spiking dynamics of the MIF neuron model are fully differentiable, eliminating the need for gradient approximations that are prevalent in the spiking neural network literature. Despite the added complexity of training directly on SPICE circuit models, we achieve 97.58% accuracy on the MNIST testing dataset and 75.26% on the Fashion-MNIST testing dataset, the highest accuracies among all fully MSNNs.