Boltzmann Encoded Adversarial Machines
This work addresses a specific issue in generative modeling for researchers, but it is incremental as it builds on existing RBM and adversarial methods.
The paper tackled the problem of likelihood-based training in Restricted Boltzmann Machines (RBMs) failing to penalize models placing high probability in low-data regions, and introduced Boltzmann Encoded Adversarial Machines (BEAMs) to address this, resulting in BEAMs outperforming RBMs and GANs on multiple benchmarks.
Restricted Boltzmann Machines (RBMs) are a class of generative neural network that are typically trained to maximize a log-likelihood objective function. We argue that likelihood-based training strategies may fail because the objective does not sufficiently penalize models that place a high probability in regions where the training data distribution has low probability. To overcome this problem, we introduce Boltzmann Encoded Adversarial Machines (BEAMs). A BEAM is an RBM trained against an adversary that uses the hidden layer activations of the RBM to discriminate between the training data and the probability distribution generated by the model. We present experiments demonstrating that BEAMs outperform RBMs and GANs on multiple benchmarks.