Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies
This work provides a gradient-free training method for SNNs, which is significant for enabling on-chip learning on neuromorphic hardware, particularly for researchers and developers working with energy-efficient AI systems.
The authors tackled the challenge of training Spiking Neural Networks (SNNs) without gradients, which is difficult due to non-differentiable spike thresholds and the high computational cost of traditional Evolution Strategies (ES). They achieved 79.21% test accuracy on N-MNIST using a low-rank factorization of ES perturbations (EGGROLL), reducing per-generation wall-clock time by 2.23x compared to full-rank ES.
Spiking Neural Networks (SNNs) offer compelling energy efficiency on neuromorphic hardware, yet their training remains challenging because the discrete spike threshold is non-differentiable. Surrogate-gradient methods sidestep this by approximating the derivative, but they impose backpropagation infrastructure that is incompatible with on-chip learning. Evolution Strategies (\es) are a natural gradient-free alternative, yet their computational cost scales with the number of parameters, making them impractical for large weight matrices. We present a method for training SNNs using EGGROLL, a low-rank factorisation of ES perturbations that reduces per-generation memory from $\mathcal{O}(mn)$ to $\mathcal{O}(r(m{+}n))$. Combining EGGROLL with a Leaky Integrate-and-Fire SNN on N-MNIST, we demonstrate that gradient-free training achieves 79.21% test accuracy while reducing per-generation wall-clock time by 2.23$\times$ relative to full-rank ES. Our results demonstrate EGGROLL is viable for SNN training, with a clear accuracy-speed tradeoff, compatible with training on neuromorphic hardware without surrogate gradients.