Restricted Boltzmann Machine, recent advances and mean-field theory
This paper provides a theoretical understanding of RBMs for researchers interested in the statistical physics underpinnings of machine learning models.
This review analyzes Restricted Boltzmann Machines (RBMs) through the lens of statistical physics, identifying a "compositional phase" where a few features combine into complex patterns. It also discusses recent mean-field based learning algorithms and ensemble dynamics equations that reproduce aspects of the learning process.
This review deals with Restricted Boltzmann Machine (RBM) under the light of statistical physics. The RBM is a classical family of Machine learning (ML) models which played a central role in the development of deep learning. Viewing it as a Spin Glass model and exhibiting various links with other models of statistical physics, we gather recent results dealing with mean-field theory in this context. First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM leading in particular to identify a {\it compositional phase} where a small number of features or modes are combined to form complex patterns. Then we discuss recent works either able to devise mean-field based learning algorithms; either able to reproduce generic aspects of the learning process from some {\it ensemble dynamics equations} or/and from linear stability arguments.