Introducing topography in convolutional neural networks
This work addresses the need for more efficient and robust neural networks in AI applications, though it is incremental as it builds on existing CNN architectures with a new bias.
The authors tackled the problem of making CNNs more memory-efficient and robust to pruning by introducing a topographic inductive bias inspired by brain organization, achieving equivalent performance to benchmarks on vision and audio tasks while improving resistance to pruning.
Parts of the brain that carry sensory tasks are organized topographically: nearby neurons are responsive to the same properties of input signals. Thus, in this work, inspired by the neuroscience literature, we proposed a new topographic inductive bias in Convolutional Neural Networks (CNNs). To achieve this, we introduced a new topographic loss and an efficient implementation to topographically organize each convolutional layer of any CNN. We benchmarked our new method on 4 datasets and 3 models in vision and audio tasks and showed equivalent performance to all benchmarks. Besides, we also showcased the generalizability of our topographic loss with how it can be used with different topographic organizations in CNNs. Finally, we demonstrated that adding the topographic inductive bias made CNNs more resistant to pruning. Our approach provides a new avenue to obtain models that are more memory efficient while maintaining better accuracy.