Multi-representation Ensembles and Delayed SGD Updates Improve Syntax-based NMT
This work addresses syntax integration in NMT for improved translation quality, but it appears incremental as it builds on existing ensemble and training methods.
The paper tackled the challenge of incorporating target syntax into Neural Machine Translation by using multi-representation ensembles and delayed SGD updates, achieving state-of-the-art performance on a difficult Japanese-English task.
We explore strategies for incorporating target syntax into Neural Machine Translation. We specifically focus on syntax in ensembles containing multiple sentence representations. We formulate beam search over such ensembles using WFSTs, and describe a delayed SGD update training procedure that is especially effective for long representations like linearized syntax. Our approach gives state-of-the-art performance on a difficult Japanese-English task.