PAVE: Premise-Aware Validation and Editing for Retrieval-Augmented LLMs
This addresses the issue of evidence-grounded consistency in retrieval-augmented LLM systems for question answering, representing an incremental improvement.
The paper tackles the problem of retrieval-augmented language models committing to answers without checking if retrieved evidence supports them, by introducing PAVE, an inference-time validation layer that improves accuracy by up to 32.7 points on a span-grounded benchmark.
Retrieval-augmented language models can retrieve relevant evidence yet still commit to answers before explicitly checking whether the retrieved context supports the conclusion. We present PAVE (Premise-Grounded Answer Validation and Editing), an inference-time validation layer for evidence-grounded question answering. PAVE decomposes retrieved context into question-conditioned atomic facts, drafts an answer, scores how well that draft is supported by the extracted premises, and revises low-support outputs before finalization. The resulting trace makes answer commitment auditable at the level of explicit premises, support scores, and revision decisions. In controlled ablations with a fixed retriever and backbone, PAVE outperforms simpler post-retrieval baselines in two evidence-grounded QA settings, with the largest gain reaching 32.7 accuracy points on a span-grounded benchmark. We view these findings as proof-of-concept evidence that explicit premise extraction plus support-gated revision can strengthen evidence-grounded consistency in retrieval-augmented LLM systems.