Efficient coding along the visual hierarchy
For researchers in computational neuroscience and AI, this work provides a biologically plausible learning principle that may explain the data efficiency of biological vision and improve brain alignment in deep networks.
The authors developed an unsupervised learning procedure based on efficient coding that compresses inputs onto dominant modes of variation in natural images, yielding features that progress from edges to shapes. These features are recognized by humans and predict fMRI responses, and combining efficient coding with supervised fine-tuning improves brain alignment and category learning in low-data settings.
Biological visual systems learn from limited experience, unlike deep learning models that rely on millions of training images. What learning principles make this possible? We tested whether efficient coding, the idea that neural representations capture the statistical structure of natural inputs, can build a hierarchy of human-aligned visual features from limited data. We developed an unsupervised learning procedure in which each layer of a deep network compresses its inputs onto the dominant modes of variation in natural images, using only local statistics and no labels, tasks, or backpropagation. This unsupervised procedure yields features that progress from edges and colors to textures and shapes. The features of this deep efficient coding model are readily recognized by human observers and are predictive of image-evoked fMRI responses in human visual cortex. Furthermore, a hybrid learning procedure that combines efficient coding with supervised fine-tuning yields better brain alignment in low-data settings and more rapid category learning. These findings suggest that efficient coding may shape representations across the entire visual hierarchy and help explain the data efficiency of biological vision.