Can Latent Alignments Improve Autoregressive Machine Translation?
This work addresses a theoretical limitation in machine translation for researchers, showing that a promising technique is not applicable to autoregressive models, which is incremental as it clarifies existing methods rather than introducing new ones.
The paper investigated whether latent alignment objectives like CTC and AXE, which improve non-autoregressive machine translation, could also enhance autoregressive models, but found that this approach leads to degenerate models and proved it is incompatible with teacher forcing.
Latent alignment objectives such as CTC and AXE significantly improve non-autoregressive machine translation models. Can they improve autoregressive models as well? We explore the possibility of training autoregressive machine translation models with latent alignment objectives, and observe that, in practice, this approach results in degenerate models. We provide a theoretical explanation for these empirical results, and prove that latent alignment objectives are incompatible with teacher forcing.