Scalability of Bayesian Network Structure Elicitation with Large Language Models: a Novel Methodology and Comparative Analysis
This addresses the scalability challenge of Bayesian Network structure elicitation for researchers and practitioners, though it appears incremental with mixed results.
The authors tackled the problem of Bayesian Network structure elicitation using Large Language Models by proposing a novel method that initializes multiple LLMs with different experiences and uses majority voting to create the final structure. Their experiments showed that their method performs better than an existing method with one of three studied LLMs, but both methods' performance significantly decreases as BN size increases.
In this work, we propose a novel method for Bayesian Networks (BNs) structure elicitation that is based on the initialization of several LLMs with different experiences, independently querying them to create a structure of the BN, and further obtaining the final structure by majority voting. We compare the method with one alternative method on various widely and not widely known BNs of different sizes and study the scalability of both methods on them. We also propose an approach to check the contamination of BNs in LLM, which shows that some widely known BNs are inapplicable for testing the LLM usage for BNs structure elicitation. We also show that some BNs may be inapplicable for such experiments because their node names are indistinguishable. The experiments on the other BNs show that our method performs better than the existing method with one of the three studied LLMs; however, the performance of both methods significantly decreases with the increase in BN size.