Estimating the Hallucination Rate of Generative AI
This addresses the issue of reliability in generative AI for users relying on in-context learning, though it is incremental as it builds on existing Bayesian interpretations.
The paper tackles the problem of estimating hallucination rates in generative AI during in-context learning by defining hallucinations as low-likelihood responses and developing a method that requires only generating and evaluating responses, achieving empirical validation on synthetic and natural language tasks.
This paper presents a method for estimating the hallucination rate for in-context learning (ICL) with generative AI. In ICL, a conditional generative model (CGM) is prompted with a dataset and a prediction question and asked to generate a response. One interpretation of ICL assumes that the CGM computes the posterior predictive of an unknown Bayesian model, which implicitly defines a joint distribution over observable datasets and latent mechanisms. This joint distribution factorizes into two components: the model prior over mechanisms and the model likelihood of datasets given a mechanism. With this perspective, we define a hallucination as a generated response to the prediction question with low model likelihood given the mechanism. We develop a new method that takes an ICL problem and estimates the probability that a CGM will generate a hallucination. Our method only requires generating prediction questions and responses from the CGM and evaluating its response log probability. We empirically evaluate our method using large language models for synthetic regression and natural language ICL tasks.