Adapting LLMs for Minimal-edit Grammatical Error Correction
This addresses the problem of improving grammatical error correction for minimal edits, which is incremental as it builds on existing methods for a specific domain.
The paper tackles adapting decoder-only large language models for minimal-edit grammatical error correction, achieving a new state-of-the-art result on the BEA-test set with a single-model system.
Decoder-only large language models have shown superior performance in the fluency-edit English Grammatical Error Correction, but their adaptation for minimal-edit English GEC is still underexplored. To improve their effectiveness in the minimal-edit approach, we explore the error rate adaptation topic and propose a novel training schedule method. Our experiments set a new state-of-the-art result for a single-model system on the BEA-test set. We also detokenize the most common English GEC datasets to match the natural way of writing text. During the process, we find that there are errors in them. Our experiments analyze whether training on detokenized datasets impacts the results and measure the impact of the usage of the datasets with corrected erroneous examples. To facilitate reproducibility, we have released the source code used to train our models.