Sequence Level Contrastive Learning for Text Summarization
This work addresses text summarization for NLP applications, presenting an incremental improvement by adapting contrastive learning from vision to text generation.
The paper tackles supervised abstractive text summarization by proposing a contrastive learning model that treats documents, gold summaries, and generated summaries as different views to maximize similarities during training, improving over BART on three datasets and achieving better faithfulness ratings in human evaluation.
Contrastive learning models have achieved great success in unsupervised visual representation learning, which maximize the similarities between feature representations of different views of the same image, while minimize the similarities between feature representations of views of different images. In text summarization, the output summary is a shorter form of the input document and they have similar meanings. In this paper, we propose a contrastive learning model for supervised abstractive text summarization, where we view a document, its gold summary and its model generated summaries as different views of the same mean representation and maximize the similarities between them during training. We improve over a strong sequence-to-sequence text generation model (i.e., BART) on three different summarization datasets. Human evaluation also shows that our model achieves better faithfulness ratings compared to its counterpart without contrastive objectives.