Construction of a Quality Estimation Dataset for Automatic Evaluation of Japanese Grammatical Error Correction
This work addresses a gap in Japanese NLP research by enabling automatic evaluation for grammatical error correction, though it is incremental as it adapts existing methods to a new language.
The authors tackled the lack of automatic evaluation tools for Japanese grammatical error correction by creating a quality estimation dataset with manual evaluations, and they verified its usefulness through meta-evaluation.
In grammatical error correction (GEC), automatic evaluation is an important factor for research and development of GEC systems. Previous studies on automatic evaluation have demonstrated that quality estimation models built from datasets with manual evaluation can achieve high performance in automatic evaluation of English GEC without using reference sentences.. However, quality estimation models have not yet been studied in Japanese, because there are no datasets for constructing quality estimation models. Therefore, in this study, we created a quality estimation dataset with manual evaluation to build an automatic evaluation model for Japanese GEC. Moreover, we conducted a meta-evaluation to verify the dataset's usefulness in building the Japanese quality estimation model.