Improving Cross-Domain Chinese Word Segmentation with Word Embeddings
This work addresses the problem of limited annotated data for cross-domain CWS, particularly in specialized domains like novels, medicine, and patents, offering an incremental improvement over existing methods.
The paper tackles the challenge of cross-domain Chinese Word Segmentation (CWS) by proposing a semi-supervised word-based approach that uses word embeddings trained on raw target domain text, resulting in a word F-measure increase of over 3.0% on four datasets and outperforming state-of-the-art methods.
Cross-domain Chinese Word Segmentation (CWS) remains a challenge despite recent progress in neural-based CWS. The limited amount of annotated data in the target domain has been the key obstacle to a satisfactory performance. In this paper, we propose a semi-supervised word-based approach to improving cross-domain CWS given a baseline segmenter. Particularly, our model only deploys word embeddings trained on raw text in the target domain, discarding complex hand-crafted features and domain-specific dictionaries. Innovative subsampling and negative sampling methods are proposed to derive word embeddings optimized for CWS. We conduct experiments on five datasets in special domains, covering domains in novels, medicine, and patent. Results show that our model can obviously improve cross-domain CWS, especially in the segmentation of domain-specific noun entities. The word F-measure increases by over 3.0% on four datasets, outperforming state-of-the-art semi-supervised and unsupervised cross-domain CWS approaches with a large margin. We make our code and data available on Github.