RethinkCWS: Is Chinese Word Segmentation a Solved Task?
This work addresses the plateau in CWS performance for NLP researchers by providing diagnostic tools and insights, though it is incremental as it focuses on evaluation rather than new models.
The paper re-evaluates Chinese Word Segmentation (CWS) by proposing a fine-grained evaluation method to diagnose model strengths and weaknesses, and quantifies discrepancies between criteria to address negative transfer in multi-criteria learning, based on experiments with eight models and seven datasets.
The performance of the Chinese Word Segmentation (CWS) systems has gradually reached a plateau with the rapid development of deep neural networks, especially the successful use of large pre-trained models. In this paper, we take stock of what we have achieved and rethink what's left in the CWS task. Methodologically, we propose a fine-grained evaluation for existing CWS systems, which not only allows us to diagnose the strengths and weaknesses of existing models (under the in-dataset setting), but enables us to quantify the discrepancy between different criterion and alleviate the negative transfer problem when doing multi-criteria learning. Strategically, despite not aiming to propose a novel model in this paper, our comprehensive experiments on eight models and seven datasets, as well as thorough analysis, could search for some promising direction for future research. We make all codes publicly available and release an interface that can quickly evaluate and diagnose user's models: https://github.com/neulab/InterpretEval.