Meta-Learning MCMC Proposals
This work addresses the need for automated and efficient inference methods in probabilistic modeling, offering a meta-learning solution that generalizes across models, though it is incremental in applying neural networks to a known bottleneck.
The paper tackled the problem of manually constructing model-specific MCMC proposals for probabilistic inference by proposing a meta-learning approach to build generalizable neural network proposals. The result showed that these learned proposals outperformed hand-tuned samplers in open-universe Gaussian mixture models and achieved higher F1 scores in a named entity recognition task.
Effective implementations of sampling-based probabilistic inference often require manually constructed, model-specific proposals. Inspired by recent progresses in meta-learning for training learning agents that can generalize to unseen environments, we propose a meta-learning approach to building effective and generalizable MCMC proposals. We parametrize the proposal as a neural network to provide fast approximations to block Gibbs conditionals. The learned neural proposals generalize to occurrences of common structural motifs across different models, allowing for the construction of a library of learned inference primitives that can accelerate inference on unseen models with no model-specific training required. We explore several applications including open-universe Gaussian mixture models, in which our learned proposals outperform a hand-tuned sampler, and a real-world named entity recognition task, in which our sampler yields higher final F1 scores than classical single-site Gibbs sampling.