Zero-Shot Temporal Resolution Domain Adaptation for Spiking Neural Networks
This addresses a practical deployment issue for SNNs on neuromorphic devices, offering a solution for edge scenarios where fine-tuning is not possible, though it is incremental in improving adaptation techniques.
The paper tackles the problem of Spiking Neural Networks (SNNs) suffering performance drops when temporal resolution differs between training and deployment, proposing three domain adaptation methods that map neuron dynamics to State Space Models to adapt parameters without retraining. The methods outperform existing scaling approaches on audio and image datasets, enabling high accuracy on high-resolution data from efficient low-resolution training.
Spiking Neural Networks (SNNs) are biologically-inspired deep neural networks that efficiently extract temporal information while offering promising gains in terms of energy efficiency and latency when deployed on neuromorphic devices. However, SNN model parameters are sensitive to temporal resolution, leading to significant performance drops when the temporal resolution of target data at the edge is not the same with that of the pre-deployment source data used for training, especially when fine-tuning is not possible at the edge. To address this challenge, we propose three novel domain adaptation methods for adapting neuron parameters to account for the change in time resolution without re-training on target time-resolution. The proposed methods are based on a mapping between neuron dynamics in SNNs and State Space Models (SSMs); and are applicable to general neuron models. We evaluate the proposed methods under spatio-temporal data tasks, namely the audio keyword spotting datasets SHD and MSWC as well as the image classification NMINST dataset. Our methods provide an alternative to - and in majority of the cases significantly outperform - the existing reference method that simply scales the time constant. Moreover, our results show that high accuracy on high temporal resolution data can be obtained by time efficient training on lower temporal resolution data and model adaptation.