Spiking Synaptic Penalty: Appropriate Penalty Term for Energy-Efficient Spiking Neural Networks
This work addresses energy efficiency for SNNs, which is crucial for low-power AI applications, but it is incremental as it builds on existing penalty-based optimization methods.
The paper tackles the problem of increasing energy consumption in spiking neural networks (SNNs) as spike firing rates rise, by introducing a novel penalty term during training to directly optimize energy consumption without altering network architecture. Results from image classification tasks show the method reduces energy consumption while maintaining accuracy, mitigating the energy-accuracy trade-off.
Spiking neural networks (SNNs) are energy-efficient neural networks because of their spiking nature. However, as the spike firing rate of SNNs increases, the energy consumption does as well, and thus, the advantage of SNNs diminishes. Here, we tackle this problem by introducing a novel penalty term for the spiking activity into the objective function in the training phase. Our method is designed so as to optimize the energy consumption metric directly without modifying the network architecture. Therefore, the proposed method can reduce the energy consumption more than other methods while maintaining the accuracy. We conducted experiments for image classification tasks, and the results indicate the effectiveness of the proposed method, which mitigates the dilemma of the energy--accuracy trade-off.