Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterization
This work addresses the challenge of constructing Bayesian Networks, especially when data is scarce, by leveraging LLMs for probabilistic modeling, though it is incremental in combining existing methods.
The paper tackles the problem of building Bayesian Networks by using Large Language Models to approximate domain expert priors for parameterization, demonstrating that LLM-derived conditional probabilities provide meaningful results compared to baselines across 80 networks in domains like healthcare and finance.
In this work, we evaluate the potential of Large Language Models (LLMs) in building Bayesian Networks (BNs) by approximating domain expert priors. LLMs have demonstrated potential as factual knowledge bases; however, their capability to generate probabilistic knowledge about real-world events remains understudied. We explore utilizing the probabilistic knowledge inherent in LLMs to derive probability estimates for statements regarding events and their relationships within a BN. Using LLMs in this context allows for the parameterization of BNs, enabling probabilistic modeling within specific domains. Our experiments on eighty publicly available Bayesian Networks, from healthcare to finance, demonstrate that querying LLMs about the conditional probabilities of events provides meaningful results when compared to baselines, including random and uniform distributions, as well as approaches based on next-token generation probabilities. We explore how these LLM-derived distributions can serve as expert priors to refine distributions extracted from data, especially when data is scarce. Overall, this work introduces a promising strategy for automatically constructing Bayesian Networks by combining probabilistic knowledge extracted from LLMs with real-world data. Additionally, we establish the first comprehensive baseline for assessing LLM performance in extracting probabilistic knowledge.