AttaCut: A Fast and Accurate Neural Thai Word Segmenter
This work addresses the need for a fast and accurate word segmenter for Thai Natural Language Processing, which is incremental as it builds on existing deep learning approaches.
The authors tackled the problem of Thai word segmentation by proposing a neural model that uses dilated CNN filters and syllable embeddings, achieving at least 5.6x faster speed and outperforming the previous state-of-the-art system in some domains.
Word segmentation is a fundamental pre-processing step for Thai Natural Language Processing. The current off-the-shelf solutions are not benchmarked consistently, so it is difficult to compare their trade-offs. We conducted a speed and accuracy comparison of the popular systems on three different domains and found that the state-of-the-art deep learning system is slow and moreover does not use sub-word structures to guide the model. Here, we propose a fast and accurate neural Thai Word Segmenter that uses dilated CNN filters to capture the environment of each character and uses syllable embeddings as features. Our system runs at least 5.6x faster and outperforms the previous state-of-the-art system on some domains. In addition, we develop the first ML-based Thai orthographical syllable segmenter, which yields syllable embeddings to be used as features by the word segmenter.