jp-evalb: Robust Alignment-based PARSEVAL Measures
This provides a more robust evaluation tool for researchers in natural language processing, though it is incremental as it builds on existing PARSEVAL measures.
The paper tackles the problem of evaluating constituency parsing by introducing jp-evalb, an alignment-based system that addresses issues with the traditional evalb script, resulting in a more flexible and accurate framework for assessment.
We introduce an evaluation system designed to compute PARSEVAL measures, offering a viable alternative to \texttt{evalb} commonly used for constituency parsing evaluation. The widely used \texttt{evalb} script has traditionally been employed for evaluating the accuracy of constituency parsing results, albeit with the requirement for consistent tokenization and sentence boundaries. In contrast, our approach, named \texttt{jp-evalb}, is founded on an alignment method. This method aligns sentences and words when discrepancies arise. It aims to overcome several known issues associated with \texttt{evalb} by utilizing the `jointly preprocessed (JP)' alignment-based method. We introduce a more flexible and adaptive framework, ultimately contributing to a more accurate assessment of constituency parsing performance.