Attention Is All You Need for Chinese Word Segmentation
This work addresses Chinese word segmentation, a key task in natural language processing for Chinese text, with incremental improvements in speed and accuracy.
The authors tackled Chinese word segmentation by proposing a model with an attention-only stacked encoder and a lightweight decoder, achieving state-of-the-art or comparable performance with the highest segmentation speed on SIGHAN Bakeoff benchmark datasets.
Taking greedy decoding algorithm as it should be, this work focuses on further strengthening the model itself for Chinese word segmentation (CWS), which results in an even more fast and more accurate CWS model. Our model consists of an attention only stacked encoder and a light enough decoder for the greedy segmentation plus two highway connections for smoother training, in which the encoder is composed of a newly proposed Transformer variant, Gaussian-masked Directional (GD) Transformer, and a biaffine attention scorer. With the effective encoder design, our model only needs to take unigram features for scoring. Our model is evaluated on SIGHAN Bakeoff benchmark datasets. The experimental results show that with the highest segmentation speed, the proposed model achieves new state-of-the-art or comparable performance against strong baselines in terms of strict closed test setting.