Improving the Diversity of Unsupervised Paraphrasing with Embedding Outputs
This work addresses the need for more fluent and diverse unsupervised paraphrasing, which is incremental as it builds on existing multilingual and embedding-based methods.
The paper tackles the problem of generating diverse paraphrases in a zero-shot setting by introducing a multilingual model that replaces the final softmax layer with word embeddings, trained on translated parallel corpora. The result shows that this approach outperforms baselines in computational metrics and human assessments across two languages.
We present a novel technique for zero-shot paraphrase generation. The key contribution is an end-to-end multilingual paraphrasing model that is trained using translated parallel corpora to generate paraphrases into "meaning spaces" -- replacing the final softmax layer with word embeddings. This architectural modification, plus a training procedure that incorporates an autoencoding objective, enables effective parameter sharing across languages for more fluent monolingual rewriting, and facilitates fluency and diversity in generation. Our continuous-output paraphrase generation models outperform zero-shot paraphrasing baselines when evaluated on two languages using a battery of computational metrics as well as in human assessment.