Deep Restricted Boltzmann Networks
This work addresses the problem of building more powerful generative models for images in computer vision and machine learning, though it appears incremental as it builds upon existing RBM extensions.
The authors tackled the limited representation power and scalability of Restricted Boltzmann Machines (RBMs) by introducing a new multi-layer network architecture called deep restricted Boltzmann network, with a joint training method that significantly outperforms normal RBMs in image and feature quality while maintaining training efficiency.
Building a good generative model for image has long been an important topic in computer vision and machine learning. Restricted Boltzmann machine (RBM) is one of such models that is simple but powerful. However, its restricted form also has placed heavy constraints on the models representation power and scalability. Many extensions have been invented based on RBM in order to produce deeper architectures with greater power. The most famous ones among them are deep belief network, which stacks multiple layer-wise pretrained RBMs to form a hybrid model, and deep Boltzmann machine, which allows connections between hidden units to form a multi-layer structure. In this paper, we present a new method to compose RBMs to form a multi-layer network style architecture and a training method that trains all layers jointly. We call the resulted structure deep restricted Boltzmann network. We further explore the combination of convolutional RBM with the normal fully connected RBM, which is made trivial under our composition framework. Experiments show that our model can generate descent images and outperform the normal RBM significantly in terms of image quality and feature quality, without losing much efficiency for training.