WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Prediction
This addresses the limitation of manual data in word alignment, making it more practical for low-resource languages, though it is incremental as it builds on existing pre-training methods.
The paper tackles the problem of word alignment by reducing dependence on manual data, using a large-scale weakly supervised dataset for pre-training via span prediction, resulting in a new state-of-the-art with improvements of 3.3~6.1 points in F1 and 1.5~6.1 points in AER on standard benchmarks.
Most existing word alignment methods rely on manual alignment datasets or parallel corpora, which limits their usefulness. Here, to mitigate the dependence on manual data, we broaden the source of supervision by relaxing the requirement for correct, fully-aligned, and parallel sentences. Specifically, we make noisy, partially aligned, and non-parallel paragraphs. We then use such a large-scale weakly-supervised dataset for word alignment pre-training via span prediction. Extensive experiments with various settings empirically demonstrate that our approach, which is named WSPAlign, is an effective and scalable way to pre-train word aligners without manual data. When fine-tuned on standard benchmarks, WSPAlign has set a new state-of-the-art by improving upon the best-supervised baseline by 3.3~6.1 points in F1 and 1.5~6.1 points in AER. Furthermore, WSPAlign also achieves competitive performance compared with the corresponding baselines in few-shot, zero-shot and cross-lingual tests, which demonstrates that WSPAlign is potentially more practical for low-resource languages than existing methods.