Sequence Level Training with Recurrent Neural Networks
This addresses a fundamental problem in natural language processing for applications requiring robust text generation, though it is an incremental improvement over existing methods.
The paper tackles the discrepancy between training language models to predict the next word and testing them to generate entire sequences, which can lead to error accumulation, by proposing a sequence-level training algorithm that directly optimizes test-time metrics like BLEU or ROUGE. The approach outperforms strong baselines for greedy generation on three tasks and is competitive with beam search while being several times faster.
Many natural language processing applications use language models to generate text. These models are typically trained to predict the next word in a sequence, given the previous words and some context such as an image. However, at test time the model is expected to generate the entire sequence from scratch. This discrepancy makes generation brittle, as errors may accumulate along the way. We address this issue by proposing a novel sequence level training algorithm that directly optimizes the metric used at test time, such as BLEU or ROUGE. On three different tasks, our approach outperforms several strong baselines for greedy generation. The method is also competitive when these baselines employ beam search, while being several times faster.