Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality
This work addresses the trade-off between factuality and quality in summarization for NLP applications, representing an incremental advancement.
The paper tackled the problem of improving factual consistency in abstractive summarization without degrading summary quality, achieving up to 6 and 11 points of absolute improvement on FactCC for XSUM and CNN/DM datasets, respectively.
Improving factual consistency of abstractive summarization has been a widely studied topic. However, most of the prior works on training factuality-aware models have ignored the negative effect it has on summary quality. We propose EFACTSUM (i.e., Effective Factual Summarization), a candidate summary generation and ranking technique to improve summary factuality without sacrificing summary quality. We show that using a contrastive learning framework with our refined candidate summaries leads to significant gains on both factuality and similarity-based metrics. Specifically, we propose a ranking strategy in which we effectively combine two metrics, thereby preventing any conflict during training. Models trained using our approach show up to 6 points of absolute improvement over the base model with respect to FactCC on XSUM and 11 points on CNN/DM, without negatively affecting either similarity-based metrics or absractiveness.