Training Robust Spiking Neural Networks with ViewPoint Transform and SpatioTemporal Stretching
This work addresses robustness issues in SNNs for neuromorphic vision, which is an incremental improvement in a domain-specific area of AI.
The paper tackles the challenge of improving the robustness of spiking neural networks (SNNs) for event camera data by proposing a novel data augmentation method called ViewPoint Transform and SpatioTemporal Stretching (VPT-STS), which achieves a state-of-the-art accuracy of 84.4% on the DVS-CIFAR10 dataset.
Neuromorphic vision sensors (event cameras) simulate biological visual perception systems and have the advantages of high temporal resolution, less data redundancy, low power consumption, and large dynamic range. Since both events and spikes are modeled from neural signals, event cameras are inherently suitable for spiking neural networks (SNNs), which are considered promising models for artificial intelligence (AI) and theoretical neuroscience. However, the unconventional visual signals of these cameras pose a great challenge to the robustness of spiking neural networks. In this paper, we propose a novel data augmentation method, ViewPoint Transform and SpatioTemporal Stretching (VPT-STS). It improves the robustness of SNNs by transforming the rotation centers and angles in the spatiotemporal domain to generate samples from different viewpoints. Furthermore, we introduce the spatiotemporal stretching to avoid potential information loss in viewpoint transformation. Extensive experiments on prevailing neuromorphic datasets demonstrate that VPT-STS is broadly effective on multi-event representations and significantly outperforms pure spatial geometric transformations. Notably, the SNNs model with VPT-STS achieves a state-of-the-art accuracy of 84.4\% on the DVS-CIFAR10 dataset.