Markov Chain Truncation for Doubly-Intractable Inference
This addresses a bottleneck in Bayesian inference for complex models, but is incremental as it builds on existing truncation and coupling techniques.
The paper tackled the problem of obtaining unbiased estimates of 1/Z for doubly-intractable distributions like Markov Random Fields, by constructing estimators from black-box importance sampling for Z, resulting in lower variance and a higher percentage of positive estimates than previous methods.
Computing partition functions, the normalizing constants of probability distributions, is often hard. Variants of importance sampling give unbiased estimates of a normalizer Z, however, unbiased estimates of the reciprocal 1/Z are harder to obtain. Unbiased estimates of 1/Z allow Markov chain Monte Carlo sampling of "doubly-intractable" distributions, such as the parameter posterior for Markov Random Fields or Exponential Random Graphs. We demonstrate how to construct unbiased estimates for 1/Z given access to black-box importance sampling estimators for Z. We adapt recent work on random series truncation and Markov chain coupling, producing estimators with lower variance and a higher percentage of positive estimates than before. Our debiasing algorithms are simple to implement, and have some theoretical and empirical advantages over existing methods.