Extracting an English-Persian Parallel Corpus from Comparable Corpora
This addresses a data bottleneck for Persian-English SMT systems, though it is incremental as it builds on existing extraction methods.
The paper tackled the scarcity of parallel data for Persian-English machine translation by extracting about 200,000 parallel sentences from Wikipedia, which improved translation quality when added to training data.
Parallel data are an important part of a reliable Statistical Machine Translation (SMT) system. The more of these data are available, the better the quality of the SMT system. However, for some language pairs such as Persian-English, parallel sources of this kind are scarce. In this paper, a bidirectional method is proposed to extract parallel sentences from English and Persian document aligned Wikipedia. Two machine translation systems are employed to translate from Persian to English and the reverse after which an IR system is used to measure the similarity of the translated sentences. Adding the extracted sentences to the training data of the existing SMT systems is shown to improve the quality of the translation. Furthermore, the proposed method slightly outperforms the one-directional approach. The extracted corpus consists of about 200,000 sentences which have been sorted by their degree of similarity calculated by the IR system and is freely available for public access on the Web.