Query-Level Uncertainty in Large Language Models
This work addresses the need for efficient and trustworthy AI by helping LLMs avoid generating incorrect or costly responses for queries beyond their capabilities, though it is incremental as it builds on existing uncertainty estimation techniques.
The paper tackles the problem of enabling Large Language Models to detect their knowledge boundaries before generating responses, proposing a training-free method called Internal Confidence that estimates query-level uncertainty. The result shows that this method outperforms baselines in confidence quality and reduces inference costs in adaptive settings like retrieval-augmented generation and model cascading while maintaining performance.
It is important for Large Language Models (LLMs) to be aware of the boundary of their knowledge, distinguishing queries they can confidently answer from those that lie beyond their capabilities. Such awareness enables models to perform adaptive inference, such as invoking retrieval-augmented generation (RAG), engaging in slow and deep thinking, or abstaining from answering when appropriate. These mechanisms are key to developing efficient and trustworthy AI. In this work, we propose a method to detect knowledge boundaries via Query-Level Uncertainty, which estimates if a model is capable of answering a given query before generating any tokens, thus avoiding the generation cost. To this end, we propose a novel, training-free method called Internal Confidence, which leverages self-evaluations across layers and tokens to provide a reliable signal of uncertainty. Empirical studies on both factual question answering and mathematical reasoning tasks demonstrate that our Internal Confidence outperforms several baselines in quality of confidence while being computationally cheaper. Furthermore, we demonstrate its benefits in adaptive inference settings, showing that for RAG and model cascading it reduces inference costs while preserving overall performance.