Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networks
This addresses the challenge of balancing performance and computation efficiency for SNNs on edge devices, representing an incremental improvement over existing compression methods.
The paper tackles the problem of deploying complex Spiking Neural Networks (SNNs) on resource-limited edge devices by proposing an improved end-to-end Minimax optimization method for compression, achieving state-of-the-art performance on various benchmark datasets and architectures.
Brain-inspired Spiking Neural Networks (SNNs) have the characteristics of event-driven and high energy-efficient, which are different from traditional Artificial Neural Networks (ANNs) when deployed on edge devices such as neuromorphic chips. Most previous work focuses on SNNs training strategies to improve model performance and brings larger and deeper network architectures. It is difficult to deploy these complex networks on resource-limited edge devices directly. To meet such demand, people compress SNNs very cautiously to balance the performance and the computation efficiency. Existing compression methods either iteratively pruned SNNs using weights norm magnitude or formulated the problem as a sparse learning optimization. We propose an improved end-to-end Minimax optimization method for this sparse learning problem to better balance the model performance and the computation efficiency. We also demonstrate that jointly applying compression and finetuning on SNNs is better than sequentially, especially for extreme compression ratios. The compressed SNN models achieved state-of-the-art (SOTA) performance on various benchmark datasets and architectures. Our code is available at https://github.com/chenjallen/Resource-Constrained-Compression-on-SNN.