Word Alignment in the Era of Deep Learning: A Tutorial
It addresses the niche problem of word alignment for researchers and practitioners in machine translation, but is incremental as it primarily surveys existing work.
This tutorial argues for the continued relevance of word alignment in machine translation, tracing its history from statistical methods like GIZA++ to neural approaches and applications such as cross-lingual annotation.
The word alignment task, despite its prominence in the era of statistical machine translation (SMT), is niche and under-explored today. In this two-part tutorial, we argue for the continued relevance for word alignment. The first part provides a historical background to word alignment as a core component of the traditional SMT pipeline. We zero-in on GIZA++, an unsupervised, statistical word aligner with surprising longevity. Jumping forward to the era of neural machine translation (NMT), we show how insights from word alignment inspired the attention mechanism fundamental to present-day NMT. The second part shifts to a survey approach. We cover neural word aligners, showing the slow but steady progress towards surpassing GIZA++ performance. Finally, we cover the present-day applications of word alignment, from cross-lingual annotation projection, to improving translation.