Conf-Gen: Conformal Uncertainty Quantification for Generative Models
This work addresses the lack of formal uncertainty quantification in unsupervised generative models, which is crucial for safety-critical AI applications.
The paper introduces Conf-Gen, a framework that adapts conformal risk control to generative models, providing formal uncertainty guarantees for tasks like image generation, conversational AI, and AI agents. It unifies previous CP applications to LLMs and extends to new domains.
Conformal prediction (CP) and its extension, conformal risk control (CRC), are established frameworks for quantifying uncertainty in supervised machine learning through formal guarantees. However, recent breakthroughs in artificial intelligence (AI) have been driven by unsupervised generative models, such as large language models (LLMs) and image generators, which are not directly compatible with CP or CRC. In this work we introduce conformal generation (Conf-Gen), a general framework adapting CRC to generative tasks while relaxing its theoretical assumptions. Conf-Gen unifies and generalizes previous attempts to apply CP to LLMs, and extends conformal methodology to entirely new domains. We demonstrate the flexibility of Conf-Gen through some novel applications, including obtaining conformal guarantees on: image generators producing non-memorized images, conversational AI systems having asked enough clarifying questions, and the output of AI agents being correct.