Neural Machine Translation for Low Resource Languages using Bilingual Lexicon Induced from Comparable Corpora
This addresses the challenge of low-resource machine translation for languages like Hindi and Tamil, though it is incremental as it builds on existing methods for comparable corpora.
This paper tackles the problem of scarce resources for non-English languages in machine translation by automatically extracting parallel sentence pairs from multilingual Wikipedia articles using a Siamese bidirectional RNN. The harvested dataset improved BLEU scores for English-Hindi and English-Tamil translation compared to training only on limited bilingual corpora.
Resources for the non-English languages are scarce and this paper addresses this problem in the context of machine translation, by automatically extracting parallel sentence pairs from the multilingual articles available on the Internet. In this paper, we have used an end-to-end Siamese bidirectional recurrent neural network to generate parallel sentences from comparable multilingual articles in Wikipedia. Subsequently, we have showed that using the harvested dataset improved BLEU scores on both NMT and phrase-based SMT systems for the low-resource language pairs: English--Hindi and English--Tamil, when compared to training exclusively on the limited bilingual corpora collected for these language pairs.