Order matters: Distributional properties of speech to young children bootstraps learning of semantic representations
This addresses the problem of understanding language acquisition mechanisms for computational models and developmental psychology, but it is incremental as it builds on existing hypotheses and methods.
The study tested whether language acquisition depends on experiencing input in order of increasing complexity by training simple recurrent neural networks on age-ordered vs. non-ordered CHILDES data, finding that age-ordered input facilitated learning of semantic structure but not sequential structure, with this advantage disappearing when utterance boundaries were removed.
Some researchers claim that language acquisition is critically dependent on experiencing linguistic input in order of increasing complexity. We set out to test this hypothesis using a simple recurrent neural network (SRN) trained to predict word sequences in CHILDES, a 5-million-word corpus of speech directed to children. First, we demonstrated that age-ordered CHILDES exhibits a gradual increase in linguistic complexity. Next, we compared the performance of two groups of SRNs trained on CHILDES which had either been age-ordered or not. Specifically, we assessed learning of grammatical and semantic structure and showed that training on age-ordered input facilitates learning of semantic, but not of sequential structure. We found that this advantage is eliminated when the models were trained on input with utterance boundary information removed.