Interpretable Uncertainty for Adaptive Retrieval and Reasoning in Question Answering
This work addresses the need for interpretable and efficient adaptive retrieval in LLM-based QA systems, offering a practical alternative to opaque or computationally expensive methods.
The paper proposes an uncertainty-aware framework for adaptive question answering that uses explicit signals from LLM internal representations to decide when to retrieve external knowledge or apply additional reasoning. The method distinguishes between knowledge insufficiency and ambiguity/conflict, estimated in a single forward pass, enabling transparent and efficient adaptive retrieval and reasoning.
Large language models (LLMs) achieve a strong performance in question answering (QA), but remain prone to hallucinations and suffer from limited transparency. Retrieval-augmented generation (RAG) can improve factuality, yet decisions about when and how to retrieve from external resources are typically based on opaque policies or computationally inefficient multi-step prompting procedures. We propose an uncertainty-aware framework for adaptive QA based on explicit signals derived from LLM internal representations. We distinguish between knowledge insufficiency and knowledge ambiguity or conflict, and efficiently estimate these from hidden states in a single forward pass. These signals guide system behaviour: RAG is triggered when knowledge is insufficient, while additional reasoning is applied when ambiguity or conflict is high. By grounding adaptive decisions in decomposed and efficiently estimable uncertainty signals, this approach provides a transparent and practical alternative to existing retrieval and reasoning strategies supporting the design of interpretable user-facing tools.