Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference
This work addresses a bottleneck in efficient SNN design for vision tasks, offering an incremental improvement over existing spiking architectures.
The paper tackles the challenge of integrating Multiplication-Free Inference (MFI) with attention and transformer mechanisms in Spiking Neural Networks (SNNs) for high-resolution vision tasks by proposing a spiking Multi-Layer Perceptron (MLP) architecture. It achieves a top-1 accuracy of 66.39% on ImageNet-1K, surpassing spiking ResNet-34 by 2.67%, and reduces computational costs and model parameters.
Advancements in adapting deep convolution architectures for Spiking Neural Networks (SNNs) have significantly enhanced image classification performance and reduced computational burdens. However, the inability of Multiplication-Free Inference (MFI) to align with attention and transformer mechanisms, which are critical to superior performance on high-resolution vision tasks, imposing limitations on these gains. To address this, our research explores a new pathway, drawing inspiration from the progress made in Multi-Layer Perceptrons (MLPs). We propose an innovative spiking MLP architecture that uses batch normalization to retain MFI compatibility and introducing a spiking patch encoding layer to enhance local feature extraction capabilities. As a result, we establish an efficient multi-stage spiking MLP network that blends effectively global receptive fields with local feature extraction for comprehensive spike-based computation. Without relying on pre-training or sophisticated SNN training techniques, our network secures a top-1 accuracy of 66.39% on the ImageNet-1K dataset, surpassing the directly trained spiking ResNet-34 by 2.67%. Furthermore, we curtail computational costs, model parameters, and simulation steps. An expanded version of our network compares with the performance of the spiking VGG-16 network with a 71.64% top-1 accuracy, all while operating with a model capacity 2.1 times smaller. Our findings highlight the potential of our deep SNN architecture in effectively integrating global and local learning abilities. Interestingly, the trained receptive field in our network mirrors the activity patterns of cortical cells. Source codes are publicly accessible at https://github.com/EMI-Group/mixer-snn.