Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations
For practitioners deploying LLMs in safety-critical settings, this provides a more robust conformal prediction method when output-level uncertainty signals are brittle under distribution shift.
The paper tackles the problem of unreliable uncertainty quantification in LLMs under distribution shift. By using internal representations (Layer-Wise Information scores) as nonconformity scores in conformal prediction, they achieve better validity-efficiency trade-offs than text-level baselines, with clear gains under cross-domain shift.
Large language models are increasingly deployed in settings where reliability matters, yet output-level uncertainty signals such as token probabilities, entropy, and self-consistency can become brittle under calibration--deployment mismatch. Conformal prediction provides finite-sample validity under exchangeability, but its practical usefulness depends on the quality of the nonconformity score. We propose a conformal framework for LLM question answering that uses internal representations rather than output-facing statistics: specifically, we introduce Layer-Wise Information (LI) scores, which measure how conditioning on the input reshapes predictive entropy across model depth, and use them as nonconformity scores within a standard split conformal pipeline. Across closed-ended and open-domain QA benchmarks, with the clearest gains under cross-domain shift, our method achieves a better validity--efficiency trade-off than strong text-level baselines while maintaining competitive in-domain reliability at the same nominal risk level. These results suggest that internal representations can provide more informative conformal scores when surface-level uncertainty is unstable under distribution shift.