PMIndia -- A Collection of Parallel Corpora of Languages of India
This work addresses the data scarcity problem for machine translation and multilingual NLP applications in South Asian languages, though it is incremental as it focuses on data collection rather than novel methods.
The authors tackled the shortage of parallel text for South Asian languages by creating PMIndia, a publicly available corpus with up to 56,000 parallel sentences pairing 13 major Indian languages with English, and they presented initial neural machine translation results on this dataset.
Parallel text is required for building high-quality machine translation (MT) systems, as well as for other multilingual NLP applications. For many South Asian languages, such data is in short supply. In this paper, we described a new publicly available corpus (PMIndia) consisting of parallel sentences which pair 13 major languages of India with English. The corpus includes up to 56000 sentences for each language pair. We explain how the corpus was constructed, including an assessment of two different automatic sentence alignment methods, and present some initial NMT results on the corpus.