Mitigating the Learning Bias towards Repetition by Self-Contrastive Training for Open-Ended Generation
This work addresses a specific issue in text generation for NLP applications, offering an incremental improvement over existing methods.
The paper tackled the problem of pretrained language models generating repetitive text in open-ended generation by addressing a learning bias that overestimates repetition probabilities, and proposed self-contrastive training to reduce repetition while maintaining fluency on two datasets.
Despite the huge progress in myriad generation tasks, pretrained language models (LMs) such as GPT2 still tend to generate repetitive texts with maximization-based decoding algorithms for open-ended generation. We attribute their overestimation of token-level repetition probabilities to the learning bias: LMs capture simple repetitive patterns faster with the MLE loss. We propose self-contrastive training to penalize the output of a premature checkpoint of the same model when it incorrectly predicts repetition, which is shown to mitigate repetition effectively while maintaining fluency on two datasets. Furthermore, we find that LMs use longer-range dependencies to predict repetitive tokens than non-repetitive ones, which may be the cause of sentence-level repetition loops.