Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignment
This work addresses the challenge of improving cross-lingual sentence embeddings for low-resource languages, which is an incremental advancement in natural language processing.
The paper tackled the problem of under-aligned cross-lingual word representations for low-resource languages by introducing a framework with explicit word alignment and training objectives, resulting in substantial improvements in bitext retrieval tasks for eight low-resource languages.
The field of cross-lingual sentence embeddings has recently experienced significant advancements, but research concerning low-resource languages has lagged due to the scarcity of parallel corpora. This paper shows that cross-lingual word representation in low-resource languages is notably under-aligned with that in high-resource languages in current models. To address this, we introduce a novel framework that explicitly aligns words between English and eight low-resource languages, utilizing off-the-shelf word alignment models. This framework incorporates three primary training objectives: aligned word prediction and word translation ranking, along with the widely used translation ranking. We evaluate our approach through experiments on the bitext retrieval task, which demonstrate substantial improvements on sentence embeddings in low-resource languages. In addition, the competitive performance of the proposed model across a broader range of tasks in high-resource languages underscores its practicality.