LLM-BI: Towards Fully Automated Bayesian Inference with Large Language Models
This addresses the problem of requiring specialized statistical expertise for Bayesian modeling, potentially enabling automated inference pipelines for probabilistic programming, though it is incremental as a proof-of-concept focused on linear regression.
The paper tackles the challenge of automating Bayesian inference by using a Large Language Model (LLM) to specify prior distributions and likelihoods from natural language, demonstrating in experiments that LLMs can successfully elicit priors and specify entire model structures for Bayesian linear regression.
A significant barrier to the widespread adoption of Bayesian inference is the specification of prior distributions and likelihoods, which often requires specialized statistical expertise. This paper investigates the feasibility of using a Large Language Model (LLM) to automate this process. We introduce LLM-BI (Large Language Model-driven Bayesian Inference), a conceptual pipeline for automating Bayesian workflows. As a proof-of-concept, we present two experiments focused on Bayesian linear regression. In Experiment I, we demonstrate that an LLM can successfully elicit prior distributions from natural language. In Experiment II, we show that an LLM can specify the entire model structure, including both priors and the likelihood, from a single high-level problem description. Our results validate the potential of LLMs to automate key steps in Bayesian modeling, enabling the possibility of an automated inference pipeline for probabilistic programming.