Mutual Information Alleviates Hallucinations in Abstractive Summarization
This addresses hallucinations in abstractive summarization, which is a critical issue for improving reliability in text generation, though it is incremental as it builds on existing decoding methods.
The paper tackles the problem of hallucinations in abstractive summarization models by identifying high model uncertainty as a key factor and proposes a decoding strategy that uses pointwise mutual information to reduce hallucinated tokens, decreasing their probability while maintaining Rouge and BertS scores on the XSum dataset.
Despite significant progress in the quality of language generated from abstractive summarization models, these models still exhibit the tendency to hallucinate, i.e., output content not supported by the source document. A number of works have tried to fix--or at least uncover the source of--the problem with limited success. In this paper, we identify a simple criterion under which models are significantly more likely to assign more probability to hallucinated content during generation: high model uncertainty. This finding offers a potential explanation for hallucinations: models default to favoring text with high marginal probability, i.e., high-frequency occurrences in the training set, when uncertain about a continuation. It also motivates possible routes for real-time intervention during decoding to prevent such hallucinations. We propose a decoding strategy that switches to optimizing for pointwise mutual information of the source and target token--rather than purely the probability of the target token--when the model exhibits uncertainty. Experiments on the XSum dataset show that our method decreases the probability of hallucinated tokens while maintaining the Rouge and BertS scores of top-performing decoding strategies.