Efficient Learning of Discrete Graphical Models
This work addresses the challenge of efficiently learning complex discrete graphical models, which is incremental as it builds on existing frameworks but offers systematic improvements in sample complexity.
The authors tackled the problem of learning discrete graphical models with node-specific alphabets and multi-body interactions, providing the first sample-efficient method based on the Interaction Screening framework that provably recovers model structure and parameters with rigorous guarantees and improved sample complexity over prior methods.
Graphical models are useful tools for describing structured high-dimensional probability distributions. Development of efficient algorithms for learning graphical models with least amount of data remains an active research topic. Reconstruction of graphical models that describe the statistics of discrete variables is a particularly challenging problem, for which the maximum likelihood approach is intractable. In this work, we provide the first sample-efficient method based on the Interaction Screening framework that allows one to provably learn fully general discrete factor models with node-specific discrete alphabets and multi-body interactions, specified in an arbitrary basis. We identify a single condition related to model parametrization that leads to rigorous guarantees on the recovery of model structure and parameters in any error norm, and is readily verifiable for a large class of models. Importantly, our bounds make explicit distinction between parameters that are proper to the model and priors used as an input to the algorithm. Finally, we show that the Interaction Screening framework includes all models previously considered in the literature as special cases, and for which our analysis shows a systematic improvement in sample complexity.