The Score Granularity Gap in Black-Box LLM Classification: A Comparative Study of Confidence Constructions
For practitioners deploying LLMs as classifiers with human-in-the-loop, the paper reveals a practical limitation of confidence scores and offers trade-offs for choosing confidence constructions.
The paper identifies a 'score granularity gap' in black-box LLM classifiers, where single-shot verbalized confidence scores take only a few distinct values, limiting thresholding options. Through a comparison of seven confidence constructions across 25 model-dataset pairs, they find that multi-query aggregation helps weak models but can degrade strong ones, providing deployment guidance.
Large language models (LLMs) are increasingly deployed as black-box classifiers in pipelines that automate confident decisions and route uncertain ones to human review. Such selective prediction needs a confidence score that an operator can threshold at a chosen risk level. Prior work asks whether LLM confidence is well calibrated or well ranked; we ask a complementary, deployment-oriented question that has been largely overlooked: at what resolution can the score be thresholded? We call the answer the score granularity gap. Through a controlled comparison of seven ways to build a confidence score, from a single verbalized number, to token probabilities, to querying the model many times and combining the answers, across 25 model-dataset pairs (9 LLMs, 3 benchmarks), we find that single-shot verbalized confidence, once correctly converted to a class probability, ranks cases surprisingly well, yet takes only a handful of distinct values. It therefore offers an operator only a few coarse thresholds, no matter how well it ranks. We show which constructions widen this gap, at what inference cost, and with what effect on ranking, notably that multi-query aggregation helps weak models but can degrade already-strong ones. We translate these trade-offs into concrete deployment guidance.