Conformal Language Modeling via Posterior Sampling
For practitioners of LLMs, this work offers a way to reduce hallucinations while maintaining sample coherence and utility, improving upon post-hoc filtering approaches.
The paper addresses hallucinations in LLMs by proposing a method that samples from approximations to an LLM posterior conditioned on a calibrated, high-scoring region, achieving target risk control with higher downstream utility than prior conformal prediction methods.
Large Language Models remain plagued by hallucinations. Recent work has sought to tame their prevalence using statistical techniques based on conformal prediction, with both theoretical and empirical success. However, these methods operate in a post-hoc fashion, treating the sampling procedure itself as atomic and then surgically altering samples to remove hallucinated claims. This disconnect between filtering and generation can result in samples that are incoherent, inconsistent, or simply unlikely under the model itself. Moreover, post-hoc surgery is unable to shift probability mass towards more useful and helpful responses. To address these issues, we propose to instead sample from approximations to an LLM posterior, where the conditioning event corresponds to a calibrated, high-scoring region. We develop a calibration procedure tailored to the setting of conditional sequential generation that effectively identifies this region and achieves target risk control. Empirically, we apply our method to case studies focused on open-ended biography generation and mathematical problem solving; compared to prior work, we obtain the same statistical guarantees, with higher downstream utility.