Unsupervised Multilingual Word Embeddings
This work addresses the challenge of creating multilingual word embeddings for low-resource languages by eliminating the need for cross-lingual supervision, which is incremental as it builds on prior unsupervised methods but improves them by leveraging interdependencies.
The paper tackles the problem of learning multilingual word embeddings without cross-lingual supervision by proposing a framework that exploits interdependencies among all language pairs, resulting in substantial outperformance over previous unsupervised methods and even beating supervised approaches in tasks like multilingual word translation and cross-lingual word similarity.
Multilingual Word Embeddings (MWEs) represent words from multiple languages in a single distributional vector space. Unsupervised MWE (UMWE) methods acquire multilingual embeddings without cross-lingual supervision, which is a significant advantage over traditional supervised approaches and opens many new possibilities for low-resource languages. Prior art for learning UMWEs, however, merely relies on a number of independently trained Unsupervised Bilingual Word Embeddings (UBWEs) to obtain multilingual embeddings. These methods fail to leverage the interdependencies that exist among many languages. To address this shortcoming, we propose a fully unsupervised framework for learning MWEs that directly exploits the relations between all language pairs. Our model substantially outperforms previous approaches in the experiments on multilingual word translation and cross-lingual word similarity. In addition, our model even beats supervised approaches trained with cross-lingual resources.