A Method for Studying Semantic Construal in Grammatical Constructions with Interpretable Contextual Embedding Spaces
This work addresses the problem of understanding semantic interpretation in linguistics for researchers, but it is incremental as it builds on existing methods for interpretable embeddings.
The authors tackled the problem of studying semantic construal in grammatical constructions by projecting contextual word embeddings into interpretable semantic spaces, showing that words in subject position are interpreted as more agentive than in object position and nouns in AANN constructions are more measurement-like.
We study semantic construal in grammatical constructions using large language models. First, we project contextual word embeddings into three interpretable semantic spaces, each defined by a different set of psycholinguistic feature norms. We validate these interpretable spaces and then use them to automatically derive semantic characterizations of lexical items in two grammatical constructions: nouns in subject or object position within the same sentence, and the AANN construction (e.g., `a beautiful three days'). We show that a word in subject position is interpreted as more agentive than the very same word in object position, and that the nouns in the AANN construction are interpreted as more measurement-like than when in the canonical alternation. Our method can probe the distributional meaning of syntactic constructions at a templatic level, abstracted away from specific lexemes.