C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models
This addresses trustworthiness issues like hallucinations in LLMs for AI safety and reliability, offering a novel certification approach but is incremental in risk analysis.
The paper tackles the problem of trustworthiness in retrieval-augmented language models (RAG) by proposing C-RAG, a framework to certify generation risks, proving that RAG achieves lower conformal generation risks than single LLMs under non-trivial conditions, with empirical validation on four NLP datasets and retrieval models.
Despite the impressive capabilities of large language models (LLMs) across diverse applications, they still suffer from trustworthiness issues, such as hallucinations and misalignments. Retrieval-augmented language models (RAG) have been proposed to enhance the credibility of generations by grounding external knowledge, but the theoretical understandings of their generation risks remains unexplored. In this paper, we answer: 1) whether RAG can indeed lead to low generation risks, 2) how to provide provable guarantees on the generation risks of RAG and vanilla LLMs, and 3) what sufficient conditions enable RAG models to reduce generation risks. We propose C-RAG, the first framework to certify generation risks for RAG models. Specifically, we provide conformal risk analysis for RAG models and certify an upper confidence bound of generation risks, which we refer to as conformal generation risk. We also provide theoretical guarantees on conformal generation risks for general bounded risk functions under test distribution shifts. We prove that RAG achieves a lower conformal generation risk than that of a single LLM when the quality of the retrieval model and transformer is non-trivial. Our intensive empirical results demonstrate the soundness and tightness of our conformal generation risk guarantees across four widely-used NLP datasets on four state-of-the-art retrieval models.