FactCorrector: A Graph-Inspired Approach to Long-Form Factuality Correction of Large Language Models
This addresses the issue of unreliable factual outputs in LLMs for knowledge-intensive applications, representing an incremental improvement in correction methods.
The paper tackles the problem of factual errors in long-form responses generated by large language models by introducing FactCorrector, a post-hoc correction method that uses structured feedback to improve factuality without retraining, achieving significant gains in factual precision on benchmarks like VELI5.
Large language models (LLMs) are widely used in knowledge-intensive applications but often generate factually incorrect responses. A promising approach to rectify these flaws is correcting LLMs using feedback. Therefore, in this paper, we introduce FactCorrector, a new post-hoc correction method that adapts across domains without retraining and leverages structured feedback about the factuality of the original response to generate a correction. To support rigorous evaluations of factuality correction methods, we also develop the VELI5 benchmark, a novel dataset containing systematically injected factual errors and ground-truth corrections. Experiments on VELI5 and several popular long-form factuality datasets show that the FactCorrector approach significantly improves factual precision while preserving relevance, outperforming strong baselines. We release our code at https://ibm.biz/factcorrector.