Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answers
Improves reliability of LLM confidence estimates for factual question answering, addressing a critical gap in calibration for real-world queries with multiple valid answers.
Existing confidence calibration methods for LLMs fail when questions have multiple correct answers, systematically underestimating confidence. The authors introduce MACE benchmark and propose Semantic Confidence Aggregation (SCA), which achieves state-of-the-art calibration under mixed-answer settings.
Confidence calibration is essential for making large language models (LLMs) reliable, yet existing training-free methods have been primarily studied under single-answer question answering. In this paper, we show that these methods break down in the presence of multiple valid answers, where disagreement among equally correct responses leads to systematic underestimation of confidence. To enable a systematic study of this phenomenon, we introduce MACE, a benchmark of 12,000 factual questions spanning six domains with varying numbers of correct answers. Experiments across 15 representative calibration methods and four LLM families (7B-72B) reveal that while accuracy increases with answer cardinality, estimated confidence consistently decreases, causing severe miscalibration for questions with mixed answer counts. To address this issue, we propose Semantic Confidence Aggregation (SCA), which aggregates confidence over multiple high-probability sampled responses. SCA achieves state-of-the-art calibration performance under mixed-answer settings while preserving strong calibration on single-answer questions.