Linguistic Productivity in Large Language Models: Models Coerce, but do not Preempt
For computational linguistics and cognitive science, this work reveals a key limitation in LLMs' ability to learn from negative evidence, distinguishing them from human language acquisition.
The study tests whether large language models exhibit entrenchment and preemption effects in linguistic productivity, finding that models show entrenchment-driven coercion but fail to use preemption to avoid overgeneralization.
Usage-based theories of grammars posit that creative productivity of the structures of language is both bolstered and constrained by two distinct frequency signals: entrenchment, stemming from high frequency usage, and preemption, stemming from having never observed a particular linguistic structure in a context where one might expect that structure to appear. Large Language Models are also usage-based, in the sense that the structures of language are learned through exposure to vast amounts of text. Here, we test whether or not the opposing statistical forces of entrenchment and preemption also encourage and constrain linguistic productivity in LLMs. We demonstrate across model architectures that larger models recognize and can reproduce with nonce words constructional productivity (entrenchment) in cases of coercion, wherein the broader constructional context coerces an atypical interpretation of a lexical item. However, we also show that even the largest models do not extend negative evidence to novel language, and statistical preemption does not enable models to avoid overgeneralization of patterns that are semantically felicitous, but never observed in data.