Brain Decodes Deep Nets
This provides a novel tool for neuroscientists and AI researchers to understand deep networks through brain analogies, though it's incremental in applying existing brain encoding techniques to model analysis.
The researchers tackled the problem of visualizing and analyzing large pre-trained vision models by mapping them onto the brain using fMRI predictions, revealing that explicit brain-network mapping (FactorTopy) is crucial and showing how training methods affect hierarchical organization and scaling behavior, with brain-like networks suffering less from catastrophic forgetting after fine-tuning.
We developed a tool for visualizing and analyzing large pre-trained vision models by mapping them onto the brain, thus exposing their hidden inside. Our innovation arises from a surprising usage of brain encoding: predicting brain fMRI measurements in response to images. We report two findings. First, explicit mapping between the brain and deep-network features across dimensions of space, layers, scales, and channels is crucial. This mapping method, FactorTopy, is plug-and-play for any deep-network; with it, one can paint a picture of the network onto the brain (literally!). Second, our visualization shows how different training methods matter: they lead to remarkable differences in hierarchical organization and scaling behavior, growing with more data or network capacity. It also provides insight into fine-tuning: how pre-trained models change when adapting to small datasets. We found brain-like hierarchically organized network suffer less from catastrophic forgetting after fine-tuned.