Learned Construction Grammars Converge Across Registers Given Increased Exposure
This addresses the problem of how language models adapt to varied linguistic contexts, but it is incremental as it builds on existing grammar induction methods.
The paper investigates whether increased exposure to language data from different registers leads to converging construction grammars, finding that with more data, grammars across 12 languages converge and a core set of universal constructions remains stable.
This paper measures the impact of increased exposure on whether learned construction grammars converge onto shared representations when trained on data from different registers. Register influences the frequency of constructions, with some structures common in formal but not informal usage. We expect that a grammar induction algorithm exposed to different registers will acquire different constructions. To what degree does increased exposure lead to the convergence of register-specific grammars? The experiments in this paper simulate language learning in 12 languages (half Germanic and half Romance) with corpora representing three registers (Twitter, Wikipedia, Web). These simulations are repeated with increasing amounts of exposure, from 100k to 2 million words, to measure the impact of exposure on the convergence of grammars. The results show that increased exposure does lead to converging grammars across all languages. In addition, a shared core of register-universal constructions remains constant across increasing amounts of exposure.