Large Language Models are Effective Priors for Causal Graph Discovery
This work addresses the challenge of integrating background knowledge into causal discovery for researchers, though it is incremental in leveraging existing LLMs.
The authors tackled the problem of causal graph discovery by using Large Language Models (LLMs) as low-cost sources of prior knowledge, finding that LLM priors improve performance on common-sense benchmarks, particularly for edge directionality.
Causal structure discovery from observations can be improved by integrating background knowledge provided by an expert to reduce the hypothesis space. Recently, Large Language Models (LLMs) have begun to be considered as sources of prior information given the low cost of querying them relative to a human expert. In this work, firstly, we propose a set of metrics for assessing LLM judgments for causal graph discovery independently of the downstream algorithm. Secondly, we systematically study a set of prompting designs that allows the model to specify priors about the structure of the causal graph. Finally, we present a general methodology for the integration of LLM priors in graph discovery algorithms, finding that they help improve performance on common-sense benchmarks and especially when used for assessing edge directionality. Our work highlights the potential as well as the shortcomings of the use of LLMs in this problem space.