Entropy Sentinel: Continuous LLM Accuracy Monitoring from Decoding Entropy Traces in STEM
This addresses the challenge of scalable monitoring and data acquisition prioritization for LLM deployments, particularly in STEM domains, though it is incremental as it applies an existing method (entropy-based statistics) to a new monitoring context.
The paper tackled the problem of monitoring LLM accuracy under domain shift by using output-entropy profiles from decoding to predict instance correctness and estimate domain-level accuracy, achieving results that often track held-out benchmark accuracy across ten STEM reasoning benchmarks and nine LLMs.
Deploying LLMs raises two coupled challenges: (1) monitoring - estimating where a model underperforms as traffic and domains drift - and (2) improvement - prioritizing data acquisition to close the largest performance gaps. We test whether an inference-time signal can estimate slice-level accuracy under domain shift. For each response, we compute an output-entropy profile from final-layer next-token probabilities (from top-k logprobs) and summarize it with eleven statistics. A lightweight classifier predicts instance correctness, and averaging predicted probabilities yields a domain-level accuracy estimate. We evaluate on ten STEM reasoning benchmarks with exhaustive train/test compositions (k in {1,2,3,4}; all "10 choose k" combinations), across nine LLMs from six families (3B-20B). Estimates often track held-out benchmark accuracy, and several models show near-monotonic ordering of domains. Output-entropy profiles are thus an accessible signal for scalable monitoring and for targeting data acquisition.