Self-organized MT Direction Maps Emerge from Spatiotemporal Contrastive Optimization
For computational neuroscience, this work unifies the computational origins of ventral and dorsal stream topographies, establishing a general mechanism for cortical self-organization.
This work presents a spatiotemporal Topographic Deep Artificial Neural Network (TDANN) that, when trained on naturalistic videos via self-supervised contrastive learning with a spatial loss, spontaneously generates brain-like direction-selective maps and pinwheel structures in the model's middle temporal (MT) area. The model quantitatively matches in vivo macaque MT physiological baselines, including direction selectivity index, circular variance, and pinwheel density.
The spatial and functional organization of the primate visual cortex is a fundamental problem in neuroscience. While recent computational frameworks like the Topographic Deep Artificial Neural Network (TDANN) have successfully modeled spatial organization in the ventral stream, the computational origins of the dorsal stream's distinct topographies, such as direction-selective maps in the middle temporal (MT) area, remain largely unresolved. In this work, we present a spatiotemporal TDANN to investigate whether MT topography is governed by the same universal principles. By training a 3D ResNet on naturalistic videos via a Momentum Contrast (MoCo) self-supervised paradigm alongside a biologically inspired spatial loss, we demonstrate the spontaneous emergence of brain-like direction maps and topological pinwheel structures. Crucially, we reveal that MT tuning properties, characterized by strong direction selectivity paired with a residual axial component, arise from a strict optimization trade-off between task-driven discriminative pressure and spatial regularization. The model's representations quantitatively match in vivo macaque MT physiological baselines, including direction selectivity index, circular variance, and pinwheel density. These findings unify the computational origins of the ventral and dorsal streams, establishing a general mechanism for cortical self-organization.