Using Contextual Information for Sentence-level Morpheme Segmentation
This work addresses morpheme segmentation for natural language processing, but it is incremental as it builds on existing methods without achieving new SOTA.
The study tackled morpheme segmentation by treating it as a sequence-to-sequence problem using entire sentences instead of isolated words, finding that a multilingual model performed better than monolingual ones but did not surpass the state-of-the-art, showing comparable results for high-resource languages but limitations with low-resource ones.
Recent advancements in morpheme segmentation primarily emphasize word-level segmentation, often neglecting the contextual relevance within the sentence. In this study, we redefine the morpheme segmentation task as a sequence-to-sequence problem, treating the entire sentence as input rather than isolating individual words. Our findings reveal that the multilingual model consistently exhibits superior performance compared to monolingual counterparts. While our model did not surpass the performance of the current state-of-the-art, it demonstrated comparable efficacy with high-resource languages while revealing limitations in low-resource language scenarios.