LightSNN: Lightweight Architecture Search for Sparse and Accurate Spiking Neural Networks
This work addresses the accuracy-efficiency trade-off in SNNs for edge devices, representing an incremental improvement with specific gains in speed and performance.
The paper tackles the problem of suboptimal performance in Spiking Neural Networks (SNNs) by developing LightSNN, a Neural Network Architecture Search (NAS) technique that balances accuracy and efficiency through sparsity enforcement. It achieves state-of-the-art results on CIFAR10 and CIFAR100, improves DVS128-Gesture performance by 4.49%, and reduces search time with a 98× speedup over SNASNet.
Spiking Neural Networks (SNNs) are highly regarded for their energy efficiency, inherent activation sparsity, and suitability for real-time processing in edge devices. However, most current SNN methods adopt architectures resembling traditional artificial neural networks (ANNs), leading to suboptimal performance when applied to SNNs. While SNNs excel in energy efficiency, they have been associated with lower accuracy levels than traditional ANNs when utilizing conventional architectures. In response, in this work we present LightSNN, a rapid and efficient Neural Network Architecture Search (NAS) technique specifically tailored for SNNs that autonomously leverages the most suitable architecture, striking a good balance between accuracy and efficiency by enforcing sparsity. Based on the spiking NAS network (SNASNet) framework, a cell-based search space including backward connections is utilized to build our training-free pruning-based NAS mechanism. Our technique assesses diverse spike activation patterns across different data samples using a sparsity-aware Hamming distance fitness evaluation. Thorough experiments are conducted on both static (CIFAR10 and CIFAR100) and neuromorphic datasets (DVS128-Gesture). Our LightSNN model achieves state-of-the-art results on CIFAR10 and CIFAR100, improves performance on DVS128Gesture by 4.49\%, and significantly reduces search time most notably offering a $98\times$ speedup over SNASNet and running 30\% faster than the best existing method on DVS128Gesture. Code is available on Github at: https://github.com/YesmineAbdennadher/LightSNN.