Decomposed Prompting Does Not Fix Knowledge Gaps, But Helps Models Say "I Don't Know"
This addresses the issue of unreliable model outputs in closed-book QA for users needing accurate and honest responses, offering a practical diagnostic tool without requiring retrieval or fine-tuning, though it is incremental as it builds on existing prompting methods.
The paper tackled the problem of large language models hallucinating confidently in closed-book question answering by investigating decomposed prompting's impact on reliability, finding that disagreements between prompting regimes serve as a precise signal for internal uncertainty, leading to a training-free abstention policy that outperforms standard uncertainty baselines in error detection, improving F1 and AUROC across settings.
Large language models often struggle to recognize their knowledge limits in closed-book question answering, leading to confident hallucinations. While decomposed prompting is typically used to improve accuracy, we investigate its impact on reliability. We evaluate three task-equivalent prompting regimes: Direct, Assistive, and Incremental, across different model scales and multi-hop QA benchmarks. We find that although accuracy gains from decomposition diminish in frontier models, disagreements between prompting regimes remain highly indicative of potential errors. Because factual knowledge is stable while hallucinations are stochastic, cross-regime agreement provides a precise signal of internal uncertainty. We leverage this signal to implement a training-free abstention policy that requires no retrieval or fine-tuning. Our results show that disagreement-based abstention outperforms standard uncertainty baselines as an error detector, improving both F1 and AUROC across settings. This demonstrates that decomposition-based prompting can serve as a practical diagnostic probe for model reliability in closed-book QA.