Continuous multilinguality with language vectors
This addresses the challenge of handling diverse and unseen language varieties in NLP, offering a more flexible approach for multilingual applications.
The paper tackled the problem of multilingual NLP by proposing continuous vector representations of language instead of discrete categories, showing they can be learned efficiently with a character-based neural language model and improve inference for unseen language varieties, with experiments on 1303 Bible translations across 990 languages demonstrating that the vectors capture genetic relationships between languages.
Most existing models for multilingual natural language processing (NLP) treat language as a discrete category, and make predictions for either one language or the other. In contrast, we propose using continuous vector representations of language. We show that these can be learned efficiently with a character-based neural language model, and used to improve inference about language varieties not seen during training. In experiments with 1303 Bible translations into 990 different languages, we empirically explore the capacity of multilingual language models, and also show that the language vectors capture genetic relationships between languages.