Rethinking LLM Uncertainty: A Multi-Agent Approach to Estimating Black-Box Model Uncertainty
This addresses reliability issues in LLM responses for applications requiring trustworthy AI, though it is an incremental improvement over self-consistency methods.
The paper tackles the problem of misleading uncertainty estimates in black-box LLMs by showing that self-consistency methods can fail due to contextual biases affecting knowledge retrieval. It introduces DiverseAgentEntropy, a multi-agent approach that improves uncertainty estimation and hallucination detection, outperforming existing techniques.
Quantifying uncertainty in black-box LLMs is vital for reliable responses and scalable oversight. Existing methods, which gauge a model's uncertainty through evaluating self-consistency in responses to the target query, can be misleading: an LLM may confidently provide an incorrect answer to a target query, yet give a confident and accurate answer to that same target query when answering a knowledge-preserving perturbation of the query. We systematically analyze the model behaviors and demonstrate that this discrepancy stems from suboptimal retrieval of parametric knowledge, often due to contextual biases that prevent consistent access to stored knowledge. We then introduce DiverseAgentEntropy, a novel, theoretically-grounded method employing multi-agent interaction across diverse query variations for uncertainty estimation of black-box LLMs. This approach more accurately assesses an LLM's true uncertainty and improves hallucination detection, outperforming existing self-consistency based techniques.