BalanceRAG: Joint Risk Calibration for Cascaded Retrieval-Augmented Generation
For developers of retrieval-augmented LLMs, BalanceRAG provides a principled way to reduce computational cost by avoiding unnecessary retrieval while maintaining reliability.
BalanceRAG introduces a joint calibration method for cascaded RAG systems that selects threshold pairs for LLM-only and RAG branches to control system-level error while maximizing coverage. Experiments on three QA benchmarks show it meets risk targets, retains more correct examples, and reduces unnecessary retrieval compared to always-on RAG.
Large language models (LLMs) can enhance factuality via retrieval-augmented generation (RAG), but applying RAG to every query is unnecessary when the model-only answer is reliable. This motivates cascaded RAG: each query is first handled by an LLM-only branch, escalated to a RAG fallback only if the primary branch is uncertain, and abstained from when neither branch is sufficiently trustworthy. However, calibrating such cascades stage by stage may be conservative, since the final utility depends on joint uncertainty thresholding of LLM-only and RAG. In this work, we develop BalanceRAG to certify threshold pairs at a target risk level. Given uncertainty scores from the two branches, BalanceRAG frames each threshold pair as an operating point on a two-dimensional lattice and identifies safe operating points using sequential graphical testing. This enables risk-adaptive threshold calibration, controlling the system-level error rate among accepted points, while retaining more examples. Furthermore, BalanceRAG extends to multi-risk calibration, allowing retrieval usage to be bounded together with the selection-conditioned risk. Experiments on three open-domain question answering (QA) benchmarks across multiple LLM backbones demonstrate that BalanceRAG meets prescribed risk levels, preserves higher coverage and more accepted correct examples, and reduces unnecessary retrieval calls compared with always-on RAG.