Hierarchical Sketch Induction for Paraphrase Generation
This work addresses the need for more varied and high-quality paraphrases in natural language processing, though it appears incremental as it builds on existing VAE methods.
The authors tackled the problem of generating syntactically diverse paraphrases by conditioning on explicit syntactic sketches, and their HRQ-VAE model produced higher-quality paraphrases than previous systems, as confirmed by extensive experiments and human evaluation.
We propose a generative model of paraphrase generation, that encourages syntactic diversity by conditioning on an explicit syntactic sketch. We introduce Hierarchical Refinement Quantized Variational Autoencoders (HRQ-VAE), a method for learning decompositions of dense encodings as a sequence of discrete latent variables that make iterative refinements of increasing granularity. This hierarchy of codes is learned through end-to-end training, and represents fine-to-coarse grained information about the input. We use HRQ-VAE to encode the syntactic form of an input sentence as a path through the hierarchy, allowing us to more easily predict syntactic sketches at test time. Extensive experiments, including a human evaluation, confirm that HRQ-VAE learns a hierarchical representation of the input space, and generates paraphrases of higher quality than previous systems.