Scaling Up Resonate-and-Fire Networks for Fast Deep Learning
This work addresses a key bottleneck in neuromorphic computing by enabling efficient deep spiking neural networks for event-based sensor data processing, representing a significant advancement over prior single-layer limitations.
The paper tackles the challenge of scaling resonate-and-fire networks for deep learning by introducing S5-RF, a structured state space model layer that enables deep spiking neural networks with up to four layers, achieving a state-of-the-art 78.8% accuracy on the Spiking Speech Commands dataset in under three hours of training.
Spiking neural networks (SNNs) present a promising computing paradigm for neuromorphic processing of event-based sensor data. The resonate-and-fire (RF) neuron, in particular, appeals through its biological plausibility, complex dynamics, yet computational simplicity. Despite theoretically predicted benefits, challenges in parameter initialization and efficient learning inhibited the implementation of RF networks, constraining their use to a single layer. In this paper, we address these shortcomings by deriving the RF neuron as a structured state space model (SSM) from the HiPPO framework. We introduce S5-RF, a new SSM layer comprised of RF neurons based on the S5 model, that features a generic initialization scheme and fast training within a deep architecture. S5-RF scales for the first time a RF network to a deep SNN with up to four layers and achieves with 78.8% a new state-of-the-art result for recurrent SNNs on the Spiking Speech Commands dataset in under three hours of training time. Moreover, compared to the reference SNNs that solve our benchmarking tasks, it achieves similar performance with much fewer spiking operations. Our code is publicly available at https://github.com/ThomasEHuber/s5-rf.