Massively Multilingual Word Embeddings
This work addresses the challenge of multilingual natural language processing for researchers and practitioners, but it is incremental as it builds on existing embedding and evaluation techniques.
The authors tackled the problem of creating and evaluating word embeddings across over fifty languages in a shared space without parallel data, achieving better correlation with downstream tasks like text categorization and parsing compared to previous methods.
We introduce new methods for estimating and evaluating embeddings of words in more than fifty languages in a single shared embedding space. Our estimation methods, multiCluster and multiCCA, use dictionaries and monolingual data; they do not require parallel data. Our new evaluation method, multiQVEC-CCA, is shown to correlate better than previous ones with two downstream tasks (text categorization and parsing). We also describe a web portal for evaluation that will facilitate further research in this area, along with open-source releases of all our methods.