Factored Verification: Detecting and Reducing Hallucination in Summaries of Academic Papers
This addresses the issue of unreliable summaries for researchers and academics, though it is incremental as it builds on existing hallucination detection benchmarks.
The paper tackles the problem of hallucination in LLM-generated summaries of academic papers by introducing Factored Verification, a method that achieves 76.2% accuracy in detecting hallucinations on the HaluEval benchmark. It finds that models like ChatGPT, GPT-4, and Claude 2 produce 0.62 to 1.55 hallucinations per summary on average, and using Factored Critiques reduces these to 0.49 to 0.95.
Hallucination plagues even frontier LLMs--but how bad is it really for summarizing academic papers? We evaluate Factored Verification, a simple automated method for detecting hallucinations in abstractive summaries. This method sets a new SotA on hallucination detection in the summarization task of the HaluEval benchmark, achieving 76.2% accuracy. We then use this method to estimate how often language models hallucinate when summarizing across multiple academic papers and find 0.62 hallucinations in the average ChatGPT (16k) summary, 0.84 for GPT-4, and 1.55 for Claude 2. We ask models to self-correct using Factored Critiques and find that this lowers the number of hallucinations to 0.49 for ChatGPT, 0.46 for GPT-4, and 0.95 for Claude 2. The hallucinations we find are often subtle, so we advise caution when using models to synthesize academic papers.