Think Global, Act Local: Bayesian Causal Discovery with Language Models in Sequential Data
This addresses the challenge of limited data and expertise in causal discovery, though it is incremental as it builds on existing Bayesian and LM methods.
The paper tackles the problem of causal discovery from sequential batch data with scarce expert knowledge by proposing BLANCE, a Bayesian framework that integrates LM-derived knowledge with observational data, outperforming prior work in structural accuracy and showing robustness to LM noise.
Causal discovery from observational data typically assumes full access to data and availability of domain experts. In practice, data often arrive in batches, and expert knowledge is scarce. Language Models (LMs) offer a surrogate but come with their own issues-hallucinations, inconsistencies, and bias. We present BLANCE (Bayesian LM-Augmented Causal Estimation)-a hybrid Bayesian framework that bridges these gaps by adaptively integrating sequential batch data with LM-derived noisy, expert knowledge while accounting for both data-induced and LM-induced biases. Our proposed representation shift from Directed Acyclic Graph (DAG) to Partial Ancestral Graph (PAG) accommodates ambiguities within a coherent Bayesian framework, allowing grounding the global LM knowledge in local observational data. To guide LM interaction, we use a sequential optimization scheme that adaptively queries the most informative edges. Across varied datasets, BLANCE outperforms prior work in structural accuracy and extends to Bayesian parameter estimation, showing robustness to LM noise.