Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Prediction
This addresses reliability issues for researchers and practitioners using LLMs to evaluate NLG, though it is incremental as it builds on existing conformal prediction methods.
The paper tackles the problem of uncertainty in LLM-as-a-judge evaluations for natural language generation by introducing a framework using conformal prediction to provide prediction intervals with coverage guarantees, achieving valid intervals as shown in experiments.
LLM-as-a-judge has become a promising paradigm for using large language models (LLMs) to evaluate natural language generation (NLG), but the uncertainty of its evaluation remains underexplored. This lack of reliability may limit its deployment in many applications. This work presents the first framework to analyze the uncertainty by offering a prediction interval of LLM-based scoring via conformal prediction. Conformal prediction constructs continuous prediction intervals from a single evaluation run, and we design an ordinal boundary adjustment for discrete rating tasks. We also suggest a midpoint-based score within the interval as a low-bias alternative to raw model score and weighted average. We perform extensive experiments and analysis, which show that conformal prediction can provide valid prediction interval with coverage guarantees. We also explore the usefulness of interval midpoint and judge reprompting for better judgment.