Joint Training Deep Boltzmann Machines for Classification
This work addresses a bottleneck in training DBMs for classification tasks, offering an incremental improvement over prior greedy or less effective methods.
The paper tackles the problem of training deep Boltzmann machines (DBMs) for classification by introducing a joint training method called multi-prediction training, which outperforms previous methods in inference accuracy and classification with missing inputs.
We introduce a new method for training deep Boltzmann machines jointly. Prior methods of training DBMs require an initial learning pass that trains the model greedily, one layer at a time, or do not perform well on classification tasks. In our approach, we train all layers of the DBM simultaneously, using a novel training procedure called multi-prediction training. The resulting model can either be interpreted as a single generative model trained to maximize a variational approximation to the generalized pseudolikelihood, or as a family of recurrent networks that share parameters and may be approximately averaged together using a novel technique we call the multi-inference trick. We show that our approach performs competitively for classification and outperforms previous methods in terms of accuracy of approximate inference and classification with missing inputs.