Exploiting Parallel Corpora to Improve Multilingual Embedding based Document and Sentence Alignment
This work addresses alignment challenges for low-resource languages like Sinhala and Tamil, offering an incremental improvement by fine-tuning pre-trained models with additional data.
The paper tackles the problem of improving multilingual sentence representations for document and sentence alignment in low-resource languages by introducing a weighting mechanism that leverages small-scale parallel corpora. Results show significant improvements in alignment for Sinhala and Tamil, with a new dataset and source code released publicly.
Multilingual sentence representations pose a great advantage for low-resource languages that do not have enough data to build monolingual models on their own. These multilingual sentence representations have been separately exploited by few research for document and sentence alignment. However, most of the low-resource languages are under-represented in these pre-trained models. Thus, in the context of low-resource languages, these models have to be fine-tuned for the task at hand, using additional data sources. This paper presents a weighting mechanism that makes use of available small-scale parallel corpora to improve the performance of multilingual sentence representations on document and sentence alignment. Experiments are conducted with respect to two low-resource languages, Sinhala and Tamil. Results on a newly created dataset of Sinhala-English, Tamil-English, and Sinhala-Tamil show that this new weighting mechanism significantly improves both document and sentence alignment. This dataset, as well as the source-code, is publicly released.