Second Language Acquisition of Neural Language Models
This addresses the problem of understanding cross-lingual transfer in AI language models, but it is incremental as it builds on prior work on first language acquisition.
The paper investigates second language (L2) acquisition in neural language models, showing that L1 pretraining accelerates linguistic generalization in L2, with language transfer configurations like L1 choice and parallel texts affecting these generalizations.
With the success of neural language models (LMs), their language acquisition has gained much attention. This work sheds light on the second language (L2) acquisition of LMs, while previous work has typically explored their first language (L1) acquisition. Specifically, we trained bilingual LMs with a scenario similar to human L2 acquisition and analyzed their cross-lingual transfer from linguistic perspectives. Our exploratory experiments demonstrated that the L1 pretraining accelerated their linguistic generalization in L2, and language transfer configurations (e.g., the L1 choice, and presence of parallel texts) substantially affected their generalizations. These clarify their (non-)human-like L2 acquisition in particular aspects.