Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks
This addresses the problem of making SNNs more competitive and energy-efficient for AI hardware implementations, representing a strong specific gain rather than a foundational breakthrough.
The paper tackled the performance gap between spiking neural networks (SNNs) and classical artificial neural networks (ANNs) by introducing a novel surrogate gradient with adaptive spiking recurrent networks, achieving state-of-the-art results for SNNs on time-domain benchmarks like speech and gesture recognition and showing computational efficiency gains of one to three orders of magnitude.
Inspired by more detailed modeling of biological neurons, Spiking neural networks (SNNs) have been investigated both as more biologically plausible and potentially more powerful models of neural computation, and also with the aim of extracting biological neurons' energy efficiency; the performance of such networks however has remained lacking compared to classical artificial neural networks (ANNs). Here, we demonstrate how a novel surrogate gradient combined with recurrent networks of tunable and adaptive spiking neurons yields state-of-the-art for SNNs on challenging benchmarks in the time-domain, like speech and gesture recognition. This also exceeds the performance of standard classical recurrent neural networks (RNNs) and approaches that of the best modern ANNs. As these SNNs exhibit sparse spiking, we show that they theoretically are one to three orders of magnitude more computationally efficient compared to RNNs with comparable performance. Together, this positions SNNs as an attractive solution for AI hardware implementations.