End-to-end topographic networks as models of cortical map formation and human visual behaviour: moving beyond convolutions
This work addresses the problem of understanding cortical map formation and human visual behavior for neuroscience and AI researchers, representing a novel method rather than an incremental improvement.
The authors tackled the limitation of convolutional neural networks ignoring topography in modeling the primate visual system by developing All-Topographic Neural Networks (All-TNNs), which emerged with features like smooth orientation maps and cortical magnification, and significantly better aligned with human behavior in object recognition compared to previous state-of-the-art models.
Computational models are an essential tool for understanding the origin and functions of the topographic organisation of the primate visual system. Yet, vision is most commonly modelled by convolutional neural networks that ignore topography by learning identical features across space. Here, we overcome this limitation by developing All-Topographic Neural Networks (All-TNNs). Trained on visual input, several features of primate topography emerge in All-TNNs: smooth orientation maps and cortical magnification in their first layer, and category-selective areas in their final layer. In addition, we introduce a novel dataset of human spatial biases in object recognition, which enables us to directly link models to behaviour. We demonstrate that All-TNNs significantly better align with human behaviour than previous state-of-the-art convolutional models due to their topographic nature. All-TNNs thereby mark an important step forward in understanding the spatial organisation of the visual brain and how it mediates visual behaviour.