Automated Analytic Asymptotic Evaluation of the Marginal Likelihood for Latent Models
This work provides a solution for researchers and practitioners in Bayesian statistics and machine learning who need efficient marginal likelihood approximations for latent variable models, though it is incremental as it builds on prior asymptotic theory.
The authors tackled the problem of analytically approximating the marginal likelihood for Bayesian networks with hidden variables, which is computationally challenging due to deviations from standard approximations like BIC. They developed and implemented two algorithms in Matlab and Maple that address dimensionality drop and singular statistics, enabling more accurate evaluations for such latent models.
We present and implement two algorithms for analytic asymptotic evaluation of the marginal likelihood of data given a Bayesian network with hidden nodes. As shown by previous work, this evaluation is particularly hard for latent Bayesian network models, namely networks that include hidden variables, where asymptotic approximation deviates from the standard BIC score. Our algorithms solve two central difficulties in asymptotic evaluation of marginal likelihood integrals, namely, evaluation of regular dimensionality drop for latent Bayesian network models and computation of non-standard approximation formulas for singular statistics for these models. The presented algorithms are implemented in Matlab and Maple and their usage is demonstrated for marginal likelihood approximations for Bayesian networks with hidden variables.