Robust Neural Abstractive Summarization Systems and Evaluation against Adversarial Information
This work addresses the issue of factual inaccuracies and adversarial vulnerabilities in abstractive summarization for natural language processing applications, representing an incremental improvement with a novel evaluation scheme.
The paper tackles the problem of neural abstractive summarization systems generating factually incorrect summaries and being vulnerable to adversarial information by proposing a semantic-aware model that learns to generate high-quality summaries through semantic interpretation. The model yields significantly better performance than the popular pointer-generator summarizer in adversarial evaluation and is confirmed by human evaluation to be more informative, faithful, and less redundant.
Sequence-to-sequence (seq2seq) neural models have been actively investigated for abstractive summarization. Nevertheless, existing neural abstractive systems frequently generate factually incorrect summaries and are vulnerable to adversarial information, suggesting a crucial lack of semantic understanding. In this paper, we propose a novel semantic-aware neural abstractive summarization model that learns to generate high quality summaries through semantic interpretation over salient content. A novel evaluation scheme with adversarial samples is introduced to measure how well a model identifies off-topic information, where our model yields significantly better performance than the popular pointer-generator summarizer. Human evaluation also confirms that our system summaries are uniformly more informative and faithful as well as less redundant than the seq2seq model.