Improving Autoregressive Training with Dynamic Oracles
This addresses training inefficiencies for NLP practitioners working on tasks like sequence tagging and text generation, though it is incremental as it builds on existing DAgger methodology.
The paper tackles the problem of exposure bias and metric mismatch in autoregressive training for NLP tasks by developing novel dynamic oracles for metrics like span-based F1, ROUGE, and BLEU, which maintain DAgger's no-regret guarantee. It shows that DAgger with dynamic oracle outperforms baseline techniques in named entity recognition and text summarization, though yields less favorable results in machine translation.
Many tasks within NLP can be framed as sequential decision problems, ranging from sequence tagging to text generation. However, for many tasks, the standard training methods, including maximum likelihood (teacher forcing) and scheduled sampling, suffer from exposure bias and a mismatch between metrics employed during training and inference. DAgger provides a solution to mitigate these problems, yet it requires a metric-specific dynamic oracle algorithm, which does not exist for many common metrics like span-based F1, ROUGE, and BLEU. In this paper, we develop these novel dynamic oracles and show they maintain DAgger's no-regret guarantee for decomposable metrics like span-based F1. We evaluate the algorithm's performance on named entity recognition (NER), text summarization, and machine translation (MT). While DAgger with dynamic oracle yields less favorable results in our MT experiments, it outperforms the baseline techniques in NER and text summarization.