When Classical Chinese Meets Machine Learning: Explaining the Relative Performances of Word and Sentence Segmentation Tasks
This work addresses text segmentation for historical Chinese documents, but it is incremental as it applies existing deep learning methods to new data.
The study tackled the problem of segmenting classical Chinese texts from the Tang Dynasty using deep learning, achieving satisfactory results and finding that differences in segmentation performance could be explained by the relative relevance among training corpora.
We consider three major text sources about the Tang Dynasty of China in our experiments that aim to segment text written in classical Chinese. These corpora include a collection of Tang Tomb Biographies, the New Tang Book, and the Old Tang Book. We show that it is possible to achieve satisfactory segmentation results with the deep learning approach. More interestingly, we found that some of the relative superiority that we observed among different designs of experiments may be explainable. The relative relevance among the training corpora provides hints/explanation for the observed differences in segmentation results that were achieved when we employed different combinations of corpora to train the classifiers.