Learning loopy graphical models with latent variables: Efficient methods and guarantees
This work addresses the challenge of learning complex graphical models with hidden variables for researchers in machine learning and statistics, offering incremental improvements in sample efficiency and theoretical guarantees.
The paper tackles the problem of structure estimation in graphical models with latent variables by developing efficient methods with provable guarantees, achieving structural consistency with sample requirements that nearly match derived lower bounds, such as n = Ω(θ_min^{-δη(η+1)-2} log p) for the Ising model.
The problem of structure estimation in graphical models with latent variables is considered. We characterize conditions for tractable graph estimation and develop efficient methods with provable guarantees. We consider models where the underlying Markov graph is locally tree-like, and the model is in the regime of correlation decay. For the special case of the Ising model, the number of samples $n$ required for structural consistency of our method scales as $n=Ω(θ_{\min}^{-δη(η+1)-2}\log p)$, where p is the number of variables, $θ_{\min}$ is the minimum edge potential, $δ$ is the depth (i.e., distance from a hidden node to the nearest observed nodes), and $η$ is a parameter which depends on the bounds on node and edge potentials in the Ising model. Necessary conditions for structural consistency under any algorithm are derived and our method nearly matches the lower bound on sample requirements. Further, the proposed method is practical to implement and provides flexibility to control the number of latent variables and the cycle lengths in the output graph.