Learning Unsupervised Word Mapping by Maximizing Mean Discrepancy
This work addresses the challenge of unstable optimization in unsupervised word mapping for cross-lingual applications, but it is incremental as it builds on existing alignment methods.
The paper tackles the problem of learning unsupervised cross-lingual word embeddings by aligning monolingual vector spaces without supervision, and the result is a model that outperforms competitive baselines by a large margin in experiments.
Cross-lingual word embeddings aim to capture common linguistic regularities of different languages, which benefit various downstream tasks ranging from machine translation to transfer learning. Recently, it has been shown that these embeddings can be effectively learned by aligning two disjoint monolingual vector spaces through a linear transformation (word mapping). In this work, we focus on learning such a word mapping without any supervision signal. Most previous work of this task adopts parametric metrics to measure distribution differences, which typically requires a sophisticated alternate optimization process, either in the form of \emph{minmax game} or intermediate \emph{density estimation}. This alternate optimization process is relatively hard and unstable. In order to avoid such sophisticated alternate optimization, we propose to learn unsupervised word mapping by directly maximizing the mean discrepancy between the distribution of transferred embedding and target embedding. Extensive experimental results show that our proposed model outperforms competitive baselines by a large margin.