Answer Only as Precisely as Justified: Calibrated Claim-Level Specificity Control for Agentic Systems
For developers of agentic systems, this provides a method to reduce overcommitment errors by locally adjusting claim specificity rather than whole-answer refusal.
Agentic systems often produce overly precise claims unsupported by evidence. The authors introduce compositional selective specificity (CSS), a post-generation layer that calibrates claim-level specificity, improving risk-utility trade-off: on LongFact, overcommitment-aware utility rose from 0.846 to 0.913 with 0.938 specificity retention.
Agentic systems often fail not by being entirely wrong, but by being too precise: a response may be generally useful while particular claims exceed what the evidence supports. We study this failure mode as overcommitment control and introduce compositional selective specificity (CSS), a post-generation layer that decomposes an answer into claims, proposes coarser backoffs, and emits each claim at the most specific calibrated level that appears admissible. The method is designed to express uncertainty as a local semantic backoff rather than as a whole-answer refusal. Across a full LongFact run and HotpotQA pilots, calibrated CSS improves the risk-utility trade-off of fixed drafts. On the full LongFact run, it raises overcommitment-aware utility from 0.846 to 0.913 relative to the no-CSS output while achieving 0.938 specificity retention. These results suggest that claim-level specificity control is a useful uncertainty interface for agentic systems and a target for future distribution-free validity layers.