Meta Learning for High-dimensional Ising Model Selection Using $\ell_1$-regularized Logistic Regression
This addresses sample efficiency in statistical learning for high-dimensional graphical models, but it is incremental as it builds on existing regularization methods with a meta-learning twist.
The paper tackles the problem of graph estimation for high-dimensional Ising models by using meta-learning with ℓ1-regularized logistic regression to reduce sample complexity for novel tasks, achieving a reduced sufficient sample complexity of Ω(d³ log d) per novel task after pooling auxiliary tasks.
In this paper, we consider the meta learning problem for estimating the graphs associated with high-dimensional Ising models, using the method of $\ell_1$-regularized logistic regression for neighborhood selection of each node. Our goal is to use the information learned from the auxiliary tasks in the learning of the novel task to reduce its sufficient sample complexity. To this end, we propose a novel generative model as well as an improper estimation method. In our setting, all the tasks are \emph{similar} in their \emph{random} model parameters and supports. By pooling all the samples from the auxiliary tasks to \emph{improperly} estimate a single parameter vector, we can recover the true support union, assumed small in size, with a high probability with a sufficient sample complexity of $Ω(1) $ per task, for $K = Ω(d^3 \log p ) $ tasks of Ising models with $p$ nodes and a maximum neighborhood size $d$. Then, with the support for the novel task restricted to the estimated support union, we prove that consistent neighborhood selection for the novel task can be obtained with a reduced sufficient sample complexity of $Ω(d^3 \log d)$.