Chinese Word Boundary Recovery through Character Alignment Projection
For NLP practitioners dealing with noisy Chinese text (e.g., learner data), this work provides a principled method to stabilize word boundary recovery, though it is incremental as it adapts existing alignment techniques.
The paper tackles Chinese word boundary recovery in non-standard text (e.g., learner errors) by proposing an alignment-based projection method that corrects over-segmentation errors, achieving improved performance over direct segmentation on a manually checked learner benchmark and a synthetic benchmark.
Chinese word segmentation is especially fragile in non-standard text, where language learner errors and other character-level divergences disrupt the word boundaries assumed by downstream annotation and evaluation. This paper formulates Chinese word boundary recovery as an alignment-based projection task. Given a noisy source sentence and a cleaner target counterpart, we first align the two strings at the character level and then project target-side word boundaries back onto the source. Beyond the recovery method itself, we introduce two evaluation resources: a manually checked learner Chinese benchmark based on MuCGEC and a controlled synthetic benchmark derived from the Chinese Penn Treebank. Experiments show that direct segmentation remains vulnerable to compound fragmentation in learner input, whereas the proposed two step projection method corrects many over-segmentation errors by using the corrected target to recover source-side word spans. The results show that word boundary recovery is distinct from ordinary segmentation and that alignment projection provides a principled mechanism for stabilizing Chinese annotation and evaluation under noisy input.