Control-DAG: Constrained Decoding for Non-Autoregressive Directed Acyclic T5 using Weighted Finite State Automata
This work addresses problems in non-autoregressive models for general Natural Language Generation, offering lexical, vocabulary, and length control, but it appears incremental as it builds on existing Directed Acyclic Transformer methods.
The paper tackled the issues of frequent Out-Of-Vocabulary errors and inability to generate entity names in Directed Acyclic Transformers for general Natural Language Generation tasks by introducing Control-DAG, a constrained decoding algorithm for DA-T5, which significantly enhanced performance on Schema Guided Dialogue and DART datasets, establishing strong non-autoregressive results for Task-Oriented Dialogue and Data-to-Text NLG.
The Directed Acyclic Transformer is a fast non-autoregressive (NAR) model that performs well in Neural Machine Translation. Two issues prevent its application to general Natural Language Generation (NLG) tasks: frequent Out-Of-Vocabulary (OOV) errors and the inability to faithfully generate entity names. We introduce Control-DAG, a constrained decoding algorithm for our Directed Acyclic T5 (DA-T5) model which offers lexical, vocabulary and length control. We show that Control-DAG significantly enhances DA-T5 on the Schema Guided Dialogue and the DART datasets, establishing strong NAR results for Task-Oriented Dialogue and Data-to-Text NLG.