MCMC Learning
This work addresses a limitation in statistical modeling for machine learning by generalizing learning concepts to more realistic MRF distributions, though it appears incremental as it builds on existing uniform distribution theory.
The paper tackles the problem of extending learning theory from the uniform distribution to Markov Random Field (MRF) distributions, establishing a novel connection between MCMC sampling properties and learning under MRFs.
The theory of learning under the uniform distribution is rich and deep, with connections to cryptography, computational complexity, and the analysis of boolean functions to name a few areas. This theory however is very limited due to the fact that the uniform distribution and the corresponding Fourier basis are rarely encountered as a statistical model. A family of distributions that vastly generalizes the uniform distribution on the Boolean cube is that of distributions represented by Markov Random Fields (MRF). Markov Random Fields are one of the main tools for modeling high dimensional data in many areas of statistics and machine learning. In this paper we initiate the investigation of extending central ideas, methods and algorithms from the theory of learning under the uniform distribution to the setup of learning concepts given examples from MRF distributions. In particular, our results establish a novel connection between properties of MCMC sampling of MRFs and learning under the MRF distribution.