Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraints
This work addresses efficiency and expressiveness trade-offs for researchers in structured NLP, but it is incremental as it builds directly on Neural QCFG.
The paper tackles the problem of expensive inference in Neural QCFG, a grammar-based seq2seq model, by proposing low-rank variants and soft constraints, resulting in improved performance over the vanilla model in most settings.
Neural QCFG is a grammar-based sequence-tosequence (seq2seq) model with strong inductive biases on hierarchical structures. It excels in interpretability and generalization but suffers from expensive inference. In this paper, we study two low-rank variants of Neural QCFG for faster inference with different trade-offs between efficiency and expressiveness. Furthermore, utilizing the symbolic interface provided by the grammar, we introduce two soft constraints over tree hierarchy and source coverage. We experiment with various datasets and find that our models outperform vanilla Neural QCFG in most settings.