ReDecode Framework for Iterative Improvement in Paraphrase Generation
This addresses the challenge of improving and rectifying errors in paraphrase generation for NLP applications like question answering and conversational systems, representing a novel method for a known bottleneck.
The paper tackles the problem of generating paraphrases by introducing an iterative refinement technique using multiple decoders, which significantly improves state-of-the-art results with over 9% and 28% absolute increases in METEOR scores on Quora and MSCOCO datasets.
Generating paraphrases, that is, different variations of a sentence conveying the same meaning, is an important yet challenging task in NLP. Automatically generating paraphrases has its utility in many NLP tasks like question answering, information retrieval, conversational systems to name a few. In this paper, we introduce iterative refinement of generated paraphrases within VAE based generation framework. Current sequence generation models lack the capability to (1) make improvements once the sentence is generated; (2) rectify errors made while decoding. We propose a technique to iteratively refine the output using multiple decoders, each one attending on the output sentence generated by the previous decoder. We improve current state of the art results significantly - with over 9% and 28% absolute increase in METEOR scores on Quora question pairs and MSCOCO datasets respectively. We also show qualitatively through examples that our re-decoding approach generates better paraphrases compared to a single decoder by rectifying errors and making improvements in paraphrase structure, inducing variations and introducing new but semantically coherent information.