PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise
This addresses the problem of improving factual consistency in LLM outputs for applications like summarization, though it is incremental as it builds on existing NLI and controlled decoding methods.
The paper tackles the problem of detecting factual inconsistencies in autoregressive text generation by generalizing entailment detection to arbitrary text prefixes, resulting in a specialized model that outperforms baseline NLI models by 5-14 F1 points and improves factual consistency in summarization, with LLaMA-3.2-3B-Instruct matching the faithfulness of an 8B model using half the memory.
Natural Language Inference (NLI) models have been used in various ways to improve the factuality of LLM outputs. This is typically done by applying an NLI model to judge whether the model output is entailed from the supposed evidence, triggering some corrective actions, such as beam reranking at inference time or RL rewards during training. While NLI models are trained to detect factual inconsistencies over complete sentences, decisions in the common autoregressive generation architecture are made for each evolving text prefix, during decoding. Addressing this setting, we generalize the entailment detection task to apply over arbitrary text prefixes, and suggest its utility for improving generation faithfulness. Providing suitable evaluation and training datasets for this task, we train MiniTruePrefixes, a novel specialized model that better detects factual inconsistencies over text prefixes, outperforming comparable baseline NLI models by 5-14 F1 points in prefix-level entailment. We further demonstrate that integrating MiniTruePrefixes into a controlled decoding framework substantially improves factual consistency in abstractive summarization. When guided by MiniTruePrefixes, LLaMA-3.2-3B-Instruct matches the faithfulness and runtime of the 8B model from the same model family, while using only half the memory.