JESC: Japanese-English Subtitle Corpus
This provides a valuable resource for researchers and practitioners in machine translation and natural language processing focusing on Japanese-English conversational data.
The authors tackled the lack of large Japanese-English parallel corpora for conversational dialogue by creating JESC, a dataset of over 3.2 million examples, which is the largest freely available of its kind.
In this paper we describe the Japanese-English Subtitle Corpus (JESC). JESC is a large Japanese-English parallel corpus covering the underrepresented domain of conversational dialogue. It consists of more than 3.2 million examples, making it the largest freely available dataset of its kind. The corpus was assembled by crawling and aligning subtitles found on the web. The assembly process incorporates a number of novel preprocessing elements to ensure high monolingual fluency and accurate bilingual alignments. We summarize its contents and evaluate its quality using human experts and baseline machine translation (MT) systems.