Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection
This work addresses the computational bottleneck in graphical model selection with unobserved variables, offering faster algorithms for large-scale problems, though it is incremental as it builds on existing convex optimization frameworks.
The authors tackled the challenge of efficiently solving a convex optimization problem for latent variable Gaussian graphical model selection, which involves decomposing an inverse covariance matrix into sparse and low-rank components. They proposed two alternating direction methods that exploit the problem's structure, achieving solutions for problems with one million variables in one to two minutes and being 5 to 35 times faster than a state-of-the-art Newton-CG proximal point algorithm.
Chandrasekaran, Parrilo and Willsky (2010) proposed a convex optimization problem to characterize graphical model selection in the presence of unobserved variables. This convex optimization problem aims to estimate an inverse covariance matrix that can be decomposed into a sparse matrix minus a low-rank matrix from sample data. Solving this convex optimization problem is very challenging, especially for large problems. In this paper, we propose two alternating direction methods for solving this problem. The first method is to apply the classical alternating direction method of multipliers to solve the problem as a consensus problem. The second method is a proximal gradient based alternating direction method of multipliers. Our methods exploit and take advantage of the special structure of the problem and thus can solve large problems very efficiently. Global convergence result is established for the proposed methods. Numerical results on both synthetic data and gene expression data show that our methods usually solve problems with one million variables in one to two minutes, and are usually five to thirty five times faster than a state-of-the-art Newton-CG proximal point algorithm.