Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization
This addresses hallucinations in abstractive summarization for NLP applications, offering a novel approach to enhance factuality by leveraging external knowledge, though it is incremental in building on prior work.
The paper tackles the problem of content hallucinations in abstractive summarization by showing that many entities in gold references are not directly in the source but require external world knowledge, and it demonstrates that using an external knowledge base improves faithfulness and factuality of summaries without making them more extractive.
Despite recent advances in abstractive summarization, current summarization systems still suffer from content hallucinations where models generate text that is either irrelevant or contradictory to the source document. However, prior work has been predicated on the assumption that any generated facts not appearing explicitly in the source are undesired hallucinations. Methods have been proposed to address this scenario by ultimately improving `faithfulness' to the source document, but in reality, there is a large portion of entities in the gold reference targets that are not directly in the source. In this work, we show that these entities are not aberrations, but they instead require utilizing external world knowledge to infer reasoning paths from entities in the source. We show that by utilizing an external knowledge base, we can improve the faithfulness of summaries without simply making them more extractive, and additionally, we show that external knowledge bases linked from the source can benefit the factuality of generated summaries.