BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data
This work provides a resource for researchers in multilingual pretraining and cognitive modeling, though it is incremental as it focuses on data curation and baseline establishment.
The authors tackled the problem of creating developmentally plausible multilingual training data by curating BabyBabelLM, a collection covering 45 languages with 100M English word equivalents per language, and they trained baseline models to establish benchmarks.
We present BabyBabelLM, a multilingual collection of datasets modeling the language a person observes from birth until they acquire a native language. We curate developmentally plausible pretraining data aiming to cover the equivalent of 100M English words of content in each of 45 languages. We compile evaluation suites and train baseline models in each language. BabyBabelLM aims to facilitate multilingual pretraining and cognitive modeling.