A Variational Approximation for Bayesian Networks with Discrete and Continuous Latent Variables
This work addresses computational challenges in Bayesian network inference for researchers and practitioners in probabilistic modeling, though it appears incremental as it builds on existing variational methods.
The authors tackled approximate inference in Bayesian networks with discrete nodes and continuous parents by using a variational approximation to the logistic function, converting it to a Gaussian for exact inference and iteratively refining parameters. They demonstrated experimentally that this method is faster and potentially more accurate than sampling, with a new evidence-handling technique enabling arbitrary distributions on observed nodes and speedups in networks with high-cardinality discrete variables.
We show how to use a variational approximation to the logistic function to perform approximate inference in Bayesian networks containing discrete nodes with continuous parents. Essentially, we convert the logistic function to a Gaussian, which facilitates exact inference, and then iteratively adjust the variational parameters to improve the quality of the approximation. We demonstrate experimentally that this approximation is faster and potentially more accurate than sampling. We also introduce a simple new technique for handling evidence, which allows us to handle arbitrary distributions on observed nodes, as well as achieving a significant speedup in networks with discrete variables of large cardinality.