Opening the black box of language acquisition
This work addresses the challenge of making language acquisition models more interpretable and biologically inspired, though it is incremental as it focuses on toy languages rather than real-world applications.
The authors tackled the problem of understanding how grammatical information is represented in language acquisition by proposing a transparent, cognitively plausible architecture based on sequence memory and chunking, which successfully learned artificial languages from scratch and extracted grammatical information.
Recent advances in large language models using deep learning techniques have renewed interest on how languages can be learned from data. However, it is unclear whether or how these models represent grammatical information from the learned languages. In addition, the models must be pre-trained on large corpora before they can be used. In this work, we propose an alternative, more transparent and cognitively plausible architecture for learning language. Instead of using deep learning, our approach uses a minimal cognitive architecture based on sequence memory and chunking. The learning mechanism is based on the principles of reinforcement learning. We test our architecture on a number of natural-like toy languages. Results show that the model can learn these artificial languages from scratch and extract grammatical information that supports learning. Our study demonstrates the power of this simple architecture and stresses the importance of sequence memory as a key component of the language learning process. Since other animals do not seem to have a faithful sequence memory, this may explain why only humans have developed complex languages.