Non-autoregressive Transformer by Position Learning
This addresses a key bottleneck in non-autoregressive models for text generation, offering improvements in efficiency and performance for tasks like translation and paraphrasing.
The paper tackled the problem of position modeling in non-autoregressive text generation by proposing PNAT, which incorporates positions as a latent variable, achieving top results on machine translation and paraphrase generation tasks.
Non-autoregressive models are promising on various text generation tasks. Previous work hardly considers to explicitly model the positions of generated words. However, position modeling is an essential problem in non-autoregressive text generation. In this study, we propose PNAT, which incorporates positions as a latent variable into the text generative process. Experimental results show that PNAT achieves top results on machine translation and paraphrase generation tasks, outperforming several strong baselines.