Training Robust Spiking Neural Networks on Neuromorphic Data with Spatiotemporal Fragments
This work addresses the need for effective data augmentation in neuromorphic vision for researchers and practitioners in event-based computing, though it is incremental as it builds on existing methods for SNNs.
The paper tackles the challenge of training robust spiking neural networks (SNNs) on neuromorphic vision data by proposing the Event SpatioTemporal Fragments (ESTF) augmentation method, which simulates brightness variations to improve robustness and achieves a state-of-the-art accuracy of 83.9% on the CIFAR10-DVS dataset.
Neuromorphic vision sensors (event cameras) are inherently suitable for spiking neural networks (SNNs) and provide novel neuromorphic vision data for this biomimetic model. Due to the spatiotemporal characteristics, novel data augmentations are required to process the unconventional visual signals of these cameras. In this paper, we propose a novel Event SpatioTemporal Fragments (ESTF) augmentation method. It preserves the continuity of neuromorphic data by drifting or inverting fragments of the spatiotemporal event stream to simulate the disturbance of brightness variations, leading to more robust spiking neural networks. Extensive experiments are performed on prevailing neuromorphic datasets. It turns out that ESTF provides substantial improvements over pure geometric transformations and outperforms other event data augmentation methods. It is worth noting that the SNNs with ESTF achieve the state-of-the-art accuracy of 83.9\% on the CIFAR10-DVS dataset.