Why all roads don't lead to Rome: Representation geometry varies across the human visual cortical hierarchy
This work addresses the efficiency-robustness tradeoff in hierarchical systems like the brain and AI, providing insights into representation geometry, but it is incremental as it builds on existing frameworks.
The study investigated how representation geometry varies across the human visual cortical hierarchy and in artificial neural networks, finding that scale-free geometry is not universal and depends on computational objectives, with specific higher-order visual areas and fine-tuned ANNs lacking this property.
Biological and artificial intelligence systems navigate the fundamental efficiency-robustness tradeoff for optimal encoding, i.e., they must efficiently encode numerous attributes of the input space while also being robust to noise. This challenge is particularly evident in hierarchical processing systems like the human brain. With a view towards understanding how systems navigate the efficiency-robustness tradeoff, we turned to a population geometry framework for analyzing representations in the human visual cortex alongside artificial neural networks (ANNs). In the ventral visual stream, we found general-purpose, scale-free representations characterized by a power law-decaying eigenspectrum in most areas. However, in certain higher-order visual areas did not have scale-free representations, indicating that scale-free geometry is not a universal property of the brain. In parallel, ANNs trained with a self-supervised learning objective also exhibited free-free geometry, but not after fine-tune on a specific task. Based on these empirical results and our analytical insights, we posit that a system's representation geometry is not a universal property and instead depends upon the computational objective.