JaQuAD: Japanese Question Answering Dataset for Machine Reading Comprehension
This addresses the problem of limited resources for Japanese NLP researchers by providing a new dataset, though it is incremental as it adapts an existing task to a new language.
The authors tackled the lack of annotated datasets for non-English languages in question answering by creating JaQuAD, a Japanese QA dataset with 39,696 human-annotated pairs, and achieved baseline results of 78.92% F1 and 63.38% EM.
Question Answering (QA) is a task in which a machine understands a given document and a question to find an answer. Despite impressive progress in the NLP area, QA is still a challenging problem, especially for non-English languages due to the lack of annotated datasets. In this paper, we present the Japanese Question Answering Dataset, JaQuAD, which is annotated by humans. JaQuAD consists of 39,696 extractive question-answer pairs on Japanese Wikipedia articles. We finetuned a baseline model which achieves 78.92% for F1 score and 63.38% for EM on test set. The dataset and our experiments are available at https://github.com/SkelterLabsInc/JaQuAD.