Explicit Syntactic Guidance for Neural Text Generation
This work addresses the challenge of generating more structured and controllable text for NLP applications, representing an incremental improvement over existing sequence-to-sequence models.
The authors tackled the problem of neural text generation by incorporating explicit syntactic structure, proposing a syntax-guided generation schema that uses constituency parse trees to guide decoding. Their method outperformed autoregressive baselines on paraphrase generation and machine translation tasks, showing improvements in interpretability, controllability, and diversity.
Most existing text generation models follow the sequence-to-sequence paradigm. Generative Grammar suggests that humans generate natural language texts by learning language grammar. We propose a syntax-guided generation schema, which generates the sequence guided by a constituency parse tree in a top-down direction. The decoding process can be decomposed into two parts: (1) predicting the infilling texts for each constituent in the lexicalized syntax context given the source sentence; (2) mapping and expanding each constituent to construct the next-level syntax context. Accordingly, we propose a structural beam search method to find possible syntax structures hierarchically. Experiments on paraphrase generation and machine translation show that the proposed method outperforms autoregressive baselines, while also demonstrating effectiveness in terms of interpretability, controllability, and diversity.