From Monte Carlo to Las Vegas: Improving Restricted Boltzmann Machine Training Through Stopping Sets
This addresses the problem of improving training efficiency and reliability for generative models like RBMs, representing an incremental advancement over existing MCMC methods.
The paper tackles the problem of training Restricted Boltzmann Machines (RBMs) by proposing a Las Vegas transformation of Markov Chain Monte Carlo estimators, called Markov Chain Las Vegas (MCLV), which provides statistical guarantees with random running times. The result shows that their MCLV-K gradient estimator (LVS-K) significantly outperforms Contrastive Divergence (CD-K) in training RBMs on the MNIST dataset.
We propose a Las Vegas transformation of Markov Chain Monte Carlo (MCMC) estimators of Restricted Boltzmann Machines (RBMs). We denote our approach Markov Chain Las Vegas (MCLV). MCLV gives statistical guarantees in exchange for random running times. MCLV uses a stopping set built from the training data and has maximum number of Markov chain steps K (referred as MCLV-K). We present a MCLV-K gradient estimator (LVS-K) for RBMs and explore the correspondence and differences between LVS-K and Contrastive Divergence (CD-K), with LVS-K significantly outperforming CD-K training RBMs over the MNIST dataset, indicating MCLV to be a promising direction in learning generative models.