Improved Natural Language Generation via Loss Truncation
This addresses the issue of noisy data degrading language generation for NLP applications, though it is an incremental improvement over existing methods.
The paper tackles the problem of neural language models being degraded by noisy and invalid references in training data by proposing loss truncation, which removes high-loss examples to optimize distinguishability, resulting in improved factual accuracy in generated samples that match human references.
Neural language models are usually trained to match the distributional properties of a large-scale corpus by minimizing the log loss. While straightforward to optimize, this approach forces the model to reproduce all variations in the dataset, including noisy and invalid references (e.g., misannotation and hallucinated facts). Worse, the commonly used log loss is overly sensitive to such phenomena and even a small fraction of noisy data can degrade performance. In this work, we show that the distinguishability of the models and reference serves as a principled and robust alternative for handling invalid references. To optimize distinguishability, we propose loss truncation, which adaptively removes high loss examples during training. We show this is as easy to optimize as log loss and tightly bounds distinguishability under noise. Empirically, we demonstrate that loss truncation outperforms existing baselines on distinguishability on a summarization task, and show that samples generated by the loss truncation model have factual accuracy ratings that exceed those of baselines and match human references.