XOResNet: Exclusive-OR Meta-Residuals Facilitate Deep Spiking Neural Networks Learning
This work is significant for researchers and practitioners developing high-performance neuromorphic systems, offering architectural insights to overcome limitations in residual learning for SNNs.
This paper addresses challenges in training deep Spiking Neural Networks (SNNs) with residual learning, specifically spike redundancy and information loss. The authors propose XOResNet, which integrates an OR-ADD (OA) shortcut connection and XOR meta-residuals, achieving state-of-the-art performance on four datasets including Fashion-MNIST, CIFAR-10, CIFAR-100, and miniImageNet.
Spiking neural networks (SNNs) hold promise for demonstrating superior learning and representation capabilities in deep models. Given the tremendous success of ResNet in deep learning, it would naturally follow to train deep SNNs with residual learning. However, existing residual structures for constructing deep SNNs still present challenges of spike redundancy or information loss, as well as redundant learning. In the present study, we first aim to address issues of relative spike redundancy in identity mapping and information loss in non-identity mapping. To this end, we propose an OR-ADD (OA) shortcut connection to merge output spikes/currents from two branches in the residual structure. Furthermore, to mitigate redundant learning in the backbone branch of the residual structure, we introduce the concept of XOR meta-residuals, i.e., selecting pre-learning residuals using the Exclusive-OR (XOR) operation for the backbone branch. Finally, by integrating the OA shortcut and XOR meta-residuals, we devise the XOR residual block and further construct XOResNet with varying depths based on this block. Extensive experiments on four datasets, Fashion-MNIST, CIFAR-10, CIFAR-100, and miniImageNet, show that the proposed XOResNet outperforms existing state-of-the-art deep SNNs optimized via gradient descent. These results validate the effectiveness of our OA shortcut and XOR meta-residual components in overcoming fundamental limitations of residual learning in SNNs, providing new architectural insights for building high-performance neuromorphic systems.