Multi-scale Evolutionary Neural Architecture Search for Deep Spiking Neural Networks
This addresses the problem of inefficient architecture design for SNNs, which are important for energy-efficient AI applications, by introducing a novel automated approach that improves performance and transferability.
The paper tackles the challenge of automatically designing neural architectures for Spiking Neural Networks (SNNs) by proposing a Multi-Scale Evolutionary Neural Architecture Search (MSE-NAS) that simultaneously evolves neuron operations, circuit motifs, and global connectivity using a brain-inspired evaluation function. The method achieves state-of-the-art performance with shorter simulation steps on multiple datasets including CIFAR10, CIFAR100, CIFAR10-DVS, and DVS128-Gesture.
Spiking Neural Networks (SNNs) have received considerable attention not only for their superiority in energy efficiency with discrete signal processing but also for their natural suitability to integrate multi-scale biological plasticity. However, most SNNs directly adopt the structure of the well-established Deep Neural Networks (DNNs), and rarely automatically design Neural Architecture Search (NAS) for SNNs. The neural motifs topology, modular regional structure and global cross-brain region connection of the human brain are the product of natural evolution and can serve as a perfect reference for designing brain-inspired SNN architecture. In this paper, we propose a Multi-Scale Evolutionary Neural Architecture Search (MSE-NAS) for SNN, simultaneously considering micro-, meso- and macro-scale brain topologies as the evolutionary search space. MSE-NAS evolves individual neuron operation, self-organized integration of multiple circuit motifs, and global connectivity across motifs through a brain-inspired indirect evaluation function, Representational Dissimilarity Matrices (RDMs). This training-free fitness function could greatly reduce computational consumption and NAS's time, and its task-independent property enables the searched SNNs to exhibit excellent transferability on multiple datasets. Furthermore, MSE-NAS show robustness against the training method and noise. Extensive experiments demonstrate that the proposed algorithm achieves state-of-the-art (SOTA) performance with shorter simulation steps on static datasets (CIFAR10, CIFAR100) and neuromorphic datasets (CIFAR10-DVS and DVS128-Gesture). The thorough analysis also illustrates the significant performance improvement and consistent bio-interpretability deriving from the topological evolution at different scales and the RDMs fitness function.