gec-metrics: A Unified Library for Grammatical Error Correction Evaluation
This provides a standardized tool for researchers and developers in GEC to improve evaluation consistency, though it is incremental as it builds on existing metrics.
The authors tackled the problem of inconsistent evaluations in grammatical error correction (GEC) by introducing gec-metrics, a unified library that enables fair system comparisons through a consistent implementation, resulting in a tool that includes meta-evaluation, analysis, and visualization features.
We introduce gec-metrics, a library for using and developing grammatical error correction (GEC) evaluation metrics through a unified interface. Our library enables fair system comparisons by ensuring that everyone conducts evaluations using a consistent implementation. Moreover, it is designed with a strong focus on API usage, making it highly extensible. It also includes meta-evaluation functionalities and provides analysis and visualization scripts, contributing to developing GEC evaluation metrics. Our code is released under the MIT license and is also distributed as an installable package. The video is available on YouTube.