Benchmarking Large Language Model Uncertainty for Prompt Optimization
This work addresses the need for better uncertainty estimation to guide prompt optimization, but it is incremental as it focuses on benchmarking rather than proposing new methods.
The paper tackles the problem of evaluating uncertainty metrics for prompt optimization in large language models by introducing a benchmark dataset, revealing that current metrics align more with answer uncertainty than correctness uncertainty.
Prompt optimization algorithms for Large Language Models (LLMs) excel in multi-step reasoning but still lack effective uncertainty estimation. This paper introduces a benchmark dataset to evaluate uncertainty metrics, focusing on Answer, Correctness, Aleatoric, and Epistemic Uncertainty. Through analysis of models like GPT-3.5-Turbo and Meta-Llama-3.1-8B-Instruct, we show that current metrics align more with Answer Uncertainty, which reflects output confidence and diversity, rather than Correctness Uncertainty, highlighting the need for improved metrics that are optimization-objective-aware to better guide prompt optimization. Our code and dataset are available at https://github.com/0Frett/PO-Uncertainty-Benchmarking.