Auto-Encoding Variational Neural Machine Translation
This addresses the problem of improving translation accuracy and robustness for machine translation systems, though it appears incremental as it builds on existing neural and variational methods.
The paper tackles machine translation by proposing a deep generative model that jointly generates source and target sentences from a shared latent representation, using neural networks and variational inference. Experiments show it consistently outperforms standard neural machine translation in in-domain, mixed-domain, and mixed-data scenarios.
We present a deep generative model of bilingual sentence pairs for machine translation. The model generates source and target sentences jointly from a shared latent representation and is parameterised by neural networks. We perform efficient training using amortised variational inference and reparameterised gradients. Additionally, we discuss the statistical implications of joint modelling and propose an efficient approximation to maximum a posteriori decoding for fast test-time predictions. We demonstrate the effectiveness of our model in three machine translation scenarios: in-domain training, mixed-domain training, and learning from a mix of gold-standard and synthetic data. Our experiments show consistently that our joint formulation outperforms conditional modelling (i.e. standard neural machine translation) in all such scenarios.