Zero-shot Faithful Factual Error Correction
This addresses the issue of maintaining integrity in knowledge bases and preventing hallucinations in sequence-to-sequence models, offering an interpretable solution with broad applicability.
The paper tackles the problem of correcting factual errors in text by introducing a zero-shot framework that formulates questions about claims, searches for correct answers in evidence, and assesses faithfulness based on consistency with evidence, outperforming fully-supervised approaches on FEVER and SciFact datasets with more faithful outputs.
Faithfully correcting factual errors is critical for maintaining the integrity of textual knowledge bases and preventing hallucinations in sequence-to-sequence models. Drawing on humans' ability to identify and correct factual errors, we present a zero-shot framework that formulates questions about input claims, looks for correct answers in the given evidence, and assesses the faithfulness of each correction based on its consistency with the evidence. Our zero-shot framework outperforms fully-supervised approaches, as demonstrated by experiments on the FEVER and SciFact datasets, where our outputs are shown to be more faithful. More importantly, the decomposability nature of our framework inherently provides interpretability. Additionally, to reveal the most suitable metrics for evaluating factual error corrections, we analyze the correlation between commonly used metrics with human judgments in terms of three different dimensions regarding intelligibility and faithfulness.