A Seq-to-Seq Transformer Premised Temporal Convolutional Network for Chinese Word Segmentation
This work addresses Chinese word segmentation, a key task in natural language processing for Chinese language applications, but it is incremental as it builds on existing transformer and convolutional approaches to improve efficiency without surpassing accuracy benchmarks.
The authors tackled Chinese word segmentation by proposing a sequence-to-sequence transformer model with a temporal convolutional network encoder, addressing issues like parallel computing and overfitting in Bi-LSTM methods. The model achieved equivalent segmentation performance to Bi-LSTM-based methods on traditional and simplified Chinese corpora, with significant improvements in parallel computing efficiency.
The prevalent approaches of Chinese word segmentation task almost rely on the Bi-LSTM neural network. However, the methods based the Bi-LSTM have some inherent drawbacks: hard to parallel computing, little efficient in applying the Dropout method to inhibit the Overfitting and little efficient in capturing the character information at the more distant site of a long sentence for the word segmentation task. In this work, we propose a sequence-to-sequence transformer model for Chinese word segmentation, which is premised a type of convolutional neural network named temporal convolutional network. The model uses the temporal convolutional network to construct an encoder, and uses one layer of fully-connected neural network to build a decoder, and applies the Dropout method to inhibit the Overfitting, and captures the character information at the distant site of a sentence by adding the layers of the encoder, and binds Conditional Random Fields model to train parameters, and uses the Viterbi algorithm to infer the final result of the Chinese word segmentation. The experiments on traditional Chinese corpora and simplified Chinese corpora show that the performance of Chinese word segmentation of the model is equivalent to the performance of the methods based the Bi-LSTM, and the model has a tremendous growth in parallel computing than the models based the Bi-LSTM.