Reliable Decision Making via Calibration Oriented Retrieval Augmented Generation
This addresses the issue of unreliable LLM-assisted decisions for users, but it is incremental as it builds on existing RAG methods.
The paper tackles the problem of LLMs providing incorrect information in decision-making tasks by proposing CalibRAG, a retrieval method that improves calibration and accuracy, with empirical validation showing enhanced performance across datasets.
Recently, Large Language Models (LLMs) have been increasingly used to support various decision-making tasks, assisting humans in making informed decisions. However, when LLMs confidently provide incorrect information, it can lead humans to make suboptimal decisions. To prevent LLMs from generating incorrect information on topics they are unsure of and to improve the accuracy of generated content, prior works have proposed Retrieval Augmented Generation (RAG), where external documents are referenced to generate responses. However, previous RAG methods focus only on retrieving documents most relevant to the input query, without specifically aiming to ensure that the human user's decisions are well-calibrated. To address this limitation, we propose a novel retrieval method called Calibrated Retrieval-Augmented Generation (CalibRAG), which ensures that decisions informed by RAG are well-calibrated. Then we empirically validate that CalibRAG improves calibration performance as well as accuracy, compared to other baselines across various datasets.