Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks
This addresses the challenge of creating robust, interpretable neural networks for neuromorphic computing and AI, though it is incremental as it builds on existing modular training methods.
The study tackled the problem of growing brain-like anatomical modularity in recurrent neural networks (RNNs) by applying brain-inspired modular training (BIMT) to compositional cognitive tasks, resulting in co-emergence of functional and anatomical clustering with superior performance in balancing task performance and network sparsity.
Recurrent neural networks (RNNs) trained on compositional tasks can exhibit functional modularity, in which neurons can be clustered by activity similarity and participation in shared computational subtasks. Unlike brains, these RNNs do not exhibit anatomical modularity, in which functional clustering is correlated with strong recurrent coupling and spatial localization of functional clusters. Contrasting with functional modularity, which can be ephemerally dependent on the input, anatomically modular networks form a robust substrate for solving the same subtasks in the future. To examine whether it is possible to grow brain-like anatomical modularity, we apply a recent machine learning method, brain-inspired modular training (BIMT), to a network being trained to solve a set of compositional cognitive tasks. We find that functional and anatomical clustering emerge together, such that functionally similar neurons also become spatially localized and interconnected. Moreover, compared to standard $L_1$ or no regularization settings, the model exhibits superior performance by optimally balancing task performance and network sparsity. In addition to achieving brain-like organization in RNNs, our findings also suggest that BIMT holds promise for applications in neuromorphic computing and enhancing the interpretability of neural network architectures.