Navigating Brain Language Representations: A Comparative Analysis of Neural Language Models and Psychologically Plausible Models
This addresses a gap in cognitive neuroscience by showing that simpler, psychologically grounded models can better predict brain activity than complex neural models, with implications for understanding language processing in the brain.
The study compared neural language models with psychologically plausible models on predicting brain activity, finding that psychologically plausible models, especially those with embodied information, outperformed neural language models across multiple modalities and languages.
Neural language models, particularly large-scale ones, have been consistently proven to be most effective in predicting brain neural activity across a range of studies. However, previous research overlooked the comparison of these models with psychologically plausible ones. Moreover, evaluations were reliant on limited, single-modality, and English cognitive datasets. To address these questions, we conducted an analysis comparing encoding performance of various neural language models and psychologically plausible models. Our study utilized extensive multi-modal cognitive datasets, examining bilingual word and discourse levels. Surprisingly, our findings revealed that psychologically plausible models outperformed neural language models across diverse contexts, encompassing different modalities such as fMRI and eye-tracking, and spanning languages from English to Chinese. Among psychologically plausible models, the one incorporating embodied information emerged as particularly exceptional. This model demonstrated superior performance at both word and discourse levels, exhibiting robust prediction of brain activation across numerous regions in both English and Chinese.