Gromov-Wasserstein Alignment of Word Embedding Spaces
This addresses the need for efficient and tunable cross-lingual alignment tools in applications like machine translation and transfer learning, though it is incremental as it builds on existing optimal transport ideas.
The paper tackles the problem of aligning word embedding spaces across languages or domains by framing it as an optimal transport problem using the Gromov-Wasserstein distance, achieving performance comparable to state-of-the-art methods in unsupervised word translation tasks.
Cross-lingual or cross-domain correspondences play key roles in tasks ranging from machine translation to transfer learning. Recently, purely unsupervised methods operating on monolingual embeddings have become effective alignment tools. Current state-of-the-art methods, however, involve multiple steps, including heuristic post-hoc refinement strategies. In this paper, we cast the correspondence problem directly as an optimal transport (OT) problem, building on the idea that word embeddings arise from metric recovery algorithms. Indeed, we exploit the Gromov-Wasserstein distance that measures how similarities between pairs of words relate across languages. We show that our OT objective can be estimated efficiently, requires little or no tuning, and results in performance comparable with the state-of-the-art in various unsupervised word translation tasks.