Support Consistency of Direct Sparse-Change Learning in Markov Networks
This work addresses the problem of efficiently detecting structural changes in Markov networks for researchers in statistics and machine learning, offering theoretical guarantees that are incremental improvements over prior methods.
The paper tackles the problem of learning sparse structure changes between two Markov networks by directly estimating their density ratio, providing sufficient conditions for successful change detection with sample complexity bounds. It proves that true sparse changes can be consistently identified with sample sizes scaling as n_p = Ω(d^2 log(m^2+m)/2) and n_q = Ω(n_p^2) for unbounded models, improving to min(n_p, n_q) = Ω(d^2 log(m^2+m)/2) for bounded models, with an exponentially decaying error bound.
We study the problem of learning sparse structure changes between two Markov networks $P$ and $Q$. Rather than fitting two Markov networks separately to two sets of data and figuring out their differences, a recent work proposed to learn changes \emph{directly} via estimating the ratio between two Markov network models. In this paper, we give sufficient conditions for \emph{successful change detection} with respect to the sample size $n_p, n_q$, the dimension of data $m$, and the number of changed edges $d$. When using an unbounded density ratio model we prove that the true sparse changes can be consistently identified for $n_p = Ω(d^2 \log \frac{m^2+m}{2})$ and $n_q = Ω({n_p^2})$, with an exponentially decaying upper-bound on learning error. Such sample complexity can be improved to $\min(n_p, n_q) = Ω(d^2 \log \frac{m^2+m}{2})$ when the boundedness of the density ratio model is assumed. Our theoretical guarantee can be applied to a wide range of discrete/continuous Markov networks.