Long-range and hierarchical language predictions in brains and algorithms
This work addresses the discrepancy between human and AI language processing for neuroscience and AI researchers, offering incremental insights into predictive coding theory.
The study tested whether the human brain makes long-range and hierarchical language predictions, unlike deep learning models that predict adjacent words, by analyzing fMRI data from 304 subjects listening to stories. Results showed that enhancing models with long-range forecast representations improved their alignment with brain activations, revealing a prediction hierarchy in the brain.
Deep learning has recently made remarkable progress in natural language processing. Yet, the resulting algorithms remain far from competing with the language abilities of the human brain. Predictive coding theory offers a potential explanation to this discrepancy: while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions. To test this hypothesis, we analyze the fMRI brain signals of 304 subjects each listening to 70min of short stories. After confirming that the activations of deep language algorithms linearly map onto those of the brain, we show that enhancing these models with long-range forecast representations improves their brain-mapping. The results further reveal a hierarchy of predictions in the brain, whereby the fronto-parietal cortices forecast more abstract and more distant representations than the temporal cortices. Overall, this study strengthens predictive coding theory and suggests a critical role of long-range and hierarchical predictions in natural language processing.