A Simple Approach to Learning Unsupervised Multilingual Embeddings
This work addresses the problem of creating shared embedding spaces for multiple languages without supervision, which is incremental as it builds on existing techniques but simplifies the approach.
The paper tackled unsupervised learning of multilingual embeddings by proposing a simple two-stage framework that decouples word alignment and mapping, achieving robust performance in tasks like bilingual lexicon induction and multilingual dependency parsing, with improvements over existing methods especially for distant languages.
Recent progress on unsupervised learning of cross-lingual embeddings in bilingual setting has given impetus to learning a shared embedding space for several languages without any supervision. A popular framework to solve the latter problem is to jointly solve the following two sub-problems: 1) learning unsupervised word alignment between several pairs of languages, and 2) learning how to map the monolingual embeddings of every language to a shared multilingual space. In contrast, we propose a simple, two-stage framework in which we decouple the above two sub-problems and solve them separately using existing techniques. The proposed approach obtains surprisingly good performance in various tasks such as bilingual lexicon induction, cross-lingual word similarity, multilingual document classification, and multilingual dependency parsing. When distant languages are involved, the proposed solution illustrates robustness and outperforms existing unsupervised multilingual word embedding approaches. Overall, our experimental results encourage development of multi-stage models for such challenging problems.