Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations
This work addresses the challenge of efficiently projecting and assessing new languages in multilingual machine translation without costly retraining, offering a tool for researchers and practitioners in computational linguistics.
The paper tackles the problem of integrating sparse linguistic typology vectors and learned multilingual embeddings to improve language representation, achieving competitive translation accuracy in tasks that rely on language similarities, such as clustering and ranking for multilingual transfer.
Sparse language vectors from linguistic typology databases and learned embeddings from tasks like multilingual machine translation have been investigated in isolation, without analysing how they could benefit from each other's language characterisation. We propose to fuse both views using singular vector canonical correlation analysis and study what kind of information is induced from each source. By inferring typological features and language phylogenies, we observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy in tasks that require information about language similarities, such as language clustering and ranking candidates for multilingual transfer. With our method, which is also released as a tool, we can easily project and assess new languages without expensive retraining of massive multilingual or ranking models, which are major disadvantages of related approaches.