A Variational Approach for Approximating Bayesian Networks by Edge Deletion
This work addresses computational challenges in Bayesian network inference for AI and machine learning applications, but it is incremental as it builds on prior methods for edge deletion and parameter optimization.
The paper tackles the problem of approximate inference in Bayesian networks by deleting edges to reduce treewidth, proposing a new method to determine auxiliary parameters by optimizing KL-divergence between original and approximate networks. It shows that this approach relates to iterative belief propagation (IBP) and its generalizations, and applies to inference problems like MAP and nonmyopic value of information.
We consider in this paper the formulation of approximate inference in Bayesian networks as a problem of exact inference on an approximate network that results from deleting edges (to reduce treewidth). We have shown in earlier work that deleting edges calls for introducing auxiliary network parameters to compensate for lost dependencies, and proposed intuitive conditions for determining these parameters. We have also shown that our method corresponds to IBP when enough edges are deleted to yield a polytree, and corresponds to some generalizations of IBP when fewer edges are deleted. In this paper, we propose a different criteria for determining auxiliary parameters based on optimizing the KL-divergence between the original and approximate networks. We discuss the relationship between the two methods for selecting parameters, shedding new light on IBP and its generalizations. We also discuss the application of our new method to approximating inference problems which are exponential in constrained treewidth, including MAP and nonmyopic value of information.