Iterative Conditional Fitting for Gaussian Ancestral Graph Models
This work addresses the need for exploratory model selection in scenarios with potential unobserved variables, providing a useful framework for researchers in statistics and machine learning dealing with latent structures.
The paper tackles the problem of maximum likelihood estimation in Gaussian ancestral graph models, which generalize Markov random fields and Bayesian networks to handle hidden variables, by presenting the Iterative Conditional Fitting (ICF) algorithm, a novel method that fits conditional distributions while holding marginals fixed, in duality to the Iterative Proportional Fitting algorithm.
Ancestral graph models, introduced by Richardson and Spirtes (2002), generalize both Markov random fields and Bayesian networks to a class of graphs with a global Markov property that is closed under conditioning and marginalization. By design, ancestral graphs encode precisely the conditional independence structures that can arise from Bayesian networks with selection and unobserved (hidden/latent) variables. Thus, ancestral graph models provide a potentially very useful framework for exploratory model selection when unobserved variables might be involved in the data-generating process but no particular hidden structure can be specified. In this paper, we present the Iterative Conditional Fitting (ICF) algorithm for maximum likelihood estimation in Gaussian ancestral graph models. The name reflects that in each step of the procedure a conditional distribution is estimated, subject to constraints, while a marginal distribution is held fixed. This approach is in duality to the well-known Iterative Proportional Fitting algorithm, in which marginal distributions are fitted while conditional distributions are held fixed.