Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output
This provides a low-cost, accessible solution for fact-checking in RAG applications, addressing hallucinations without relying on large language models, though it is incremental as it builds on existing NLI techniques.
The paper tackles the problem of detecting nonfactual outputs in retrieval-augmented generation (RAG) systems by introducing a light-weight fact-checker that uses compact natural language inference models to compute factuality scores, achieving high AUC across multiple datasets.
We present a light-weight approach for detecting nonfactual outputs from retrieval-augmented generation (RAG). Given a context and putative output, we compute a factuality score that can be thresholded to yield a binary decision to check the results of LLM-based question-answering, summarization, or other systems. Unlike factuality checkers that themselves rely on LLMs, we use compact, open-source natural language inference (NLI) models that yield a freely accessible solution with low latency and low cost at run-time, and no need for LLM fine-tuning. The approach also enables downstream mitigation and correction of hallucinations, by tracing them back to specific context chunks. Our experiments show high area under the ROC curve (AUC) across a wide range of relevant open source datasets, indicating the effectiveness of our method for fact-checking RAG output.