A Neurobiologically Motivated Analysis of Distributional Semantic Models
This provides insights for researchers in computational linguistics and cognitive science on the limitations of word embeddings for modeling abstract concepts, though it is incremental as it builds on existing brain-based vector methods.
The paper tackled the problem of understanding what information is encoded in text-based word vectors by analyzing them using neurobiologically motivated brain-based vectors, finding that social and cognitive information is well-encoded while emotional information is not.
The pervasive use of distributional semantic models or word embeddings in a variety of research fields is due to their remarkable ability to represent the meanings of words for both practical application and cognitive modeling. However, little has been known about what kind of information is encoded in text-based word vectors. This lack of understanding is particularly problematic when word vectors are regarded as a model of semantic representation for abstract concepts. This paper attempts to reveal the internal information of distributional word vectors by the analysis using Binder et al.'s (2016) brain-based vectors, explicitly structured conceptual representations based on neurobiologically motivated attributes. In the analysis, the mapping from text-based vectors to brain-based vectors is trained and prediction performance is evaluated by comparing the estimated and original brain-based vectors. The analysis demonstrates that social and cognitive information is better encoded in text-based word vectors, but emotional information is not. This result is discussed in terms of embodied theories for abstract concepts.