SpiKernel: A Kernel Size Exploration Methodology for Improving Accuracy of the Embedded Spiking Neural Network Systems
This work addresses the need for high-accuracy SNNs with low memory footprint for resource-constrained embedded applications, representing an incremental improvement in model optimization.
The paper tackles the problem of Spiking Neural Networks (SNNs) requiring large model sizes for high accuracy, which is unsuitable for embedded applications, by proposing SpiKernel, a methodology that improves accuracy through kernel size exploration, achieving state-of-the-art results (e.g., 93.24% on CIFAR10) with less than 10M parameters and up to 4.8x faster search time.
Spiking Neural Networks (SNNs) can offer ultra-low power/energy consumption for machine learning-based application tasks due to their sparse spike-based operations. Currently, most of the SNN architectures need a significantly larger model size to achieve higher accuracy, which is not suitable for resource-constrained embedded applications. Therefore, developing SNNs that can achieve high accuracy with acceptable memory footprint is highly needed. Toward this, we propose SpiKernel, a novel methodology that improves the accuracy of SNNs through kernel size exploration. Its key steps include (1) investigating the impact of different kernel sizes on the accuracy, (2) devising new sets of kernel sizes, (3) generating SNN architectures using neural architecture search based on the selected kernel sizes, and (4) analyzing the accuracy-memory trade-offs for SNN model selection. The experimental results show that our SpiKernel achieves higher accuracy than state-of-the-art works (i.e., 93.24% for CIFAR10, 70.84% for CIFAR100, and 62% for TinyImageNet) with less than 10M parameters and up to 4.8x speed-up of searching time, thereby making it suitable for embedded applications.