Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines
This work addresses a fundamental issue in training energy-based models like RBMs, which is crucial for researchers in machine learning and statistical physics, though it is incremental as it builds on existing contrastive divergence methods.
The paper tackles the challenge of training Restricted Boltzmann Machines (RBMs) by analyzing the role of mixing time in Monte Carlo sampling, showing that RBMs operate in equilibrium or out-of-equilibrium regimes based on the interplay between mixing time and gradient approximation steps. It demonstrates empirically that mixing time increases during learning, leading to regime transitions, with out-of-equilibrium training using small k steps resulting in slow dynamics, while equilibrium training yields faster convergence and smoother sampling.
Training Restricted Boltzmann Machines (RBMs) has been challenging for a long time due to the difficulty of computing precisely the log-likelihood gradient. Over the past decades, many works have proposed more or less successful training recipes but without studying the crucial quantity of the problem: the mixing time, i.e. the number of Monte Carlo iterations needed to sample new configurations from a model. In this work, we show that this mixing time plays a crucial role in the dynamics and stability of the trained model, and that RBMs operate in two well-defined regimes, namely equilibrium and out-of-equilibrium, depending on the interplay between this mixing time of the model and the number of steps, $k$, used to approximate the gradient. We further show empirically that this mixing time increases with the learning, which often implies a transition from one regime to another as soon as $k$ becomes smaller than this time. In particular, we show that using the popular $k$ (persistent) contrastive divergence approaches, with $k$ small, the dynamics of the learned model are extremely slow and often dominated by strong out-of-equilibrium effects. On the contrary, RBMs trained in equilibrium display faster dynamics, and a smooth convergence to dataset-like configurations during the sampling. Finally we discuss how to exploit in practice both regimes depending on the task one aims to fulfill: (i) short $k$ can be used to generate convincing samples in short learning times, (ii) large $k$ (or increasingly large) is needed to learn the correct equilibrium distribution of the RBM. Finally, the existence of these two operational regimes seems to be a general property of energy based models trained via likelihood maximization.