Towards Efficient Deep Spiking Neural Networks Construction with Spiking Activity based Pruning
This work addresses the need for efficient SNN deployment in low-power scenarios, though it is incremental as it builds on existing pruning techniques by introducing structured and dynamic elements.
The paper tackles the problem of compressing deep spiking neural networks (SNNs) to reduce computational load and accelerate inference by proposing a structured pruning approach based on spiking channel activity, which maintains model performance while enhancing efficiency.
The emergence of deep and large-scale spiking neural networks (SNNs) exhibiting high performance across diverse complex datasets has led to a need for compressing network models due to the presence of a significant number of redundant structural units, aiming to more effectively leverage their low-power consumption and biological interpretability advantages. Currently, most model compression techniques for SNNs are based on unstructured pruning of individual connections, which requires specific hardware support. Hence, we propose a structured pruning approach based on the activity levels of convolutional kernels named Spiking Channel Activity-based (SCA) network pruning framework. Inspired by synaptic plasticity mechanisms, our method dynamically adjusts the network's structure by pruning and regenerating convolutional kernels during training, enhancing the model's adaptation to the current target task. While maintaining model performance, this approach refines the network architecture, ultimately reducing computational load and accelerating the inference process. This indicates that structured dynamic sparse learning methods can better facilitate the application of deep SNNs in low-power and high-efficiency scenarios.