Random Feature Spiking Neural Networks
This addresses the training challenge for SNNs, which are important for energy-efficient machine learning, though it is an incremental adaptation of existing methods to a new domain.
The paper tackled the difficulty of training Spiking Neural Networks (SNNs) due to non-differentiability and sparsity by adapting Random Feature Methods (RFMs) to SNNs, resulting in a novel algorithm (S-SWIM) that achieves high accuracies on time series forecasting and serves as an effective initialization strategy.
Spiking Neural Networks (SNNs) as Machine Learning (ML) models have recently received a lot of attention as a potentially more energy-efficient alternative to conventional Artificial Neural Networks. The non-differentiability and sparsity of the spiking mechanism can make these models very difficult to train with algorithms based on propagating gradients through the spiking non-linearity. We address this problem by adapting the paradigm of Random Feature Methods (RFMs) from Artificial Neural Networks (ANNs) to Spike Response Model (SRM) SNNs. This approach allows training of SNNs without approximation of the spike function gradient. Concretely, we propose a novel data-driven, fast, high-performance, and interpretable algorithm for end-to-end training of SNNs inspired by the SWIM algorithm for RFM-ANNs, which we coin S-SWIM. We provide a thorough theoretical discussion and supplementary numerical experiments showing that S-SWIM can reach high accuracies on time series forecasting as a standalone strategy and serve as an effective initialisation strategy before gradient-based training. Additional ablation studies show that our proposed method performs better than random sampling of network weights.