Building a Japanese Document-Level Relation Extraction Dataset Assisted by Cross-Lingual Transfer
This work addresses the problem of limited resources for document-level relation extraction in non-English languages, specifically Japanese, by proposing an assisted annotation approach, though it is incremental as it builds on existing English datasets and methods.
The authors tackled the lack of document-level relation extraction datasets for non-English languages by constructing a Japanese dataset using cross-lingual transfer from English, but found that models trained on it had low recalls due to translation issues. They then developed a method where annotators edit model predictions, reducing human edit steps by about 50% compared to previous approaches, and evaluated existing models on the new dataset to highlight challenges in Japanese and cross-lingual DocRE.
Document-level Relation Extraction (DocRE) is the task of extracting all semantic relationships from a document. While studies have been conducted on English DocRE, limited attention has been given to DocRE in non-English languages. This work delves into effectively utilizing existing English resources to promote DocRE studies in non-English languages, with Japanese as the representative case. As an initial attempt, we construct a dataset by transferring an English dataset to Japanese. However, models trained on such a dataset suffer from low recalls. We investigate the error cases and attribute the failure to different surface structures and semantics of documents translated from English and those written by native speakers. We thus switch to explore if the transferred dataset can assist human annotation on Japanese documents. In our proposal, annotators edit relation predictions from a model trained on the transferred dataset. Quantitative analysis shows that relation recommendations suggested by the model help reduce approximately 50% of the human edit steps compared with the previous approach. Experiments quantify the performance of existing DocRE models on our collected dataset, portraying the challenges of Japanese and cross-lingual DocRE.