LSB: Local Self-Balancing MCMC in Discrete Spaces
This addresses sampling efficiency for energy-based models and Markov networks, representing an incremental improvement over existing local MCMC methods.
The paper tackles the problem of sampling in discrete domains by introducing the Local Self-Balancing sampler (LSB), which autonomously adapts to target distributions and reduces the number of target evaluations needed for convergence. Experiments show LSB converges with fewer queries to the oracle distribution compared to recent local MCMC samplers.
We present the Local Self-Balancing sampler (LSB), a local Markov Chain Monte Carlo (MCMC) method for sampling in purely discrete domains, which is able to autonomously adapt to the target distribution and to reduce the number of target evaluations required to converge. LSB is based on (i) a parametrization of locally balanced proposals, (ii) a newly proposed objective function based on mutual information and (iii) a self-balancing learning procedure, which minimises the proposed objective to update the proposal parameters. Experiments on energy-based models and Markov networks show that LSB converges using a smaller number of queries to the oracle distribution compared to recent local MCMC samplers.