Massively Multilingual Document Alignment with Cross-lingual Sentence-Mover's Distance
This work addresses the need for aligned multilingual data for NLP tasks like machine translation, offering incremental improvements over existing methods.
The paper tackled the problem of aligning documents across languages by developing an unsupervised scoring function using cross-lingual sentence embeddings to compute semantic distances, resulting in improved alignment performance with gains of 7% on high-resource, 15% on mid-resource, and 22% on low-resource language pairs compared to baselines.
Document alignment aims to identify pairs of documents in two distinct languages that are of comparable content or translations of each other. Such aligned data can be used for a variety of NLP tasks from training cross-lingual representations to mining parallel data for machine translation. In this paper we develop an unsupervised scoring function that leverages cross-lingual sentence embeddings to compute the semantic distance between documents in different languages. These semantic distances are then used to guide a document alignment algorithm to properly pair cross-lingual web documents across a variety of low, mid, and high-resource language pairs. Recognizing that our proposed scoring function and other state of the art methods are computationally intractable for long web documents, we utilize a more tractable greedy algorithm that performs comparably. We experimentally demonstrate that our distance metric performs better alignment than current baselines outperforming them by 7% on high-resource language pairs, 15% on mid-resource language pairs, and 22% on low-resource language pairs.