Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models
This work addresses a specific problem in psycholinguistics and AI by examining incremental effects in language processing, with implications for understanding human cognition and improving language models.
The study investigated how the distance between an attractor noun and verb affects grammatical agreement in humans and neural networks, finding that both make more errors when the attractor is closer, with neural networks performing near chance level while humans mostly overcome interference, and reaction times show a linear effect of distance.
Subject-verb agreement in the presence of an attractor noun located between the main noun and the verb elicits complex behavior: judgments of grammaticality are modulated by the grammatical features of the attractor. For example, in the sentence "The girl near the boys likes climbing", the attractor (boys) disagrees in grammatical number with the verb (likes), creating a locally implausible transition probability. Here, we parametrically modulate the distance between the attractor and the verb while keeping the length of the sentence equal. We evaluate the performance of both humans and two artificial neural network models: both make more mistakes when the attractor is closer to the verb, but neural networks get close to the chance level while humans are mostly able to overcome the attractor interference. Additionally, we report a linear effect of attractor distance on reaction times. We hypothesize that a possible reason for the proximity effect is the calculation of transition probabilities between adjacent words. Nevertheless, classical models of attraction such as the cue-based model might suffice to explain this phenomenon, thus paving the way for new research. Data and analyses available at https://osf.io/d4g6k