ConfRAG: Confidence-Guided Retrieval-Augmenting Generation
This addresses the problem of factual inaccuracy and computational inefficiency in LLM-based question answering systems, representing a strong incremental improvement over existing RAG approaches.
The paper tackles the problem of LLMs hallucinating factual statements and inefficient retrieval in RAG systems by introducing ConfQA, a fine-tuning strategy that reduces hallucination rates from 20-40% to below 5%, and ConfRAG, which triggers RAG only when the model is unsure, achieving over 95% accuracy while cutting unnecessary retrievals by over 30%.
Can Large Language Models (LLMs) be trained to avoid hallucinating factual statements, and can Retrieval-Augmented Generation (RAG) be triggered only when necessary to reduce retrieval and computation costs? In this work, we address both challenges simultaneously. We introduce ConfQA, a fine-tuning strategy that reduces hallucination rates from 20-40% to below 5% across multiple factuality benchmarks. The approach is simple: when the model answers correctly, it is trained to output the answer; otherwise, it is trained to respond with "I am unsure". Two design choices make this training effective: (1) a dampening prompt ("answer only if you are confident") that explicitly discourages overconfident hallucinations, and (2) training data drawn from atomic factual statements (e.g., knowledge graph attribute values), which calibrates model confidence and yields robust generalization across domains and question types. Building on ConfQA, we propose ConfRAG, a triggering strategy that invokes RAG only when the model responses with unsure. This framework achieves accuracy above 95% in ideal case while reducing unnecessary external retrievals by over 30%.