Matrix Variate RBM and Its Applications
This work addresses a domain-specific problem for researchers and practitioners using RBMs in computer vision, offering an incremental improvement by generalizing the classic RBM to handle matrix data more efficiently.
The paper tackles the problem of vectorization in Restricted Boltzmann Machines (RBMs) for image data, which loses spatial information and increases dimensionality, by proposing a Matrix-Variate RBM (MVRBM) that models data in matrix form with bilinear transforms. The result is a model with fewer parameters, faster training, and comparable performance, demonstrated on image super-resolution and handwritten digit recognition.
Restricted Boltzmann Machine (RBM) is an importan- t generative model modeling vectorial data. While applying an RBM in practice to images, the data have to be vec- torized. This results in high-dimensional data and valu- able spatial information has got lost in vectorization. In this paper, a Matrix-Variate Restricted Boltzmann Machine (MVRBM) model is proposed by generalizing the classic RBM to explicitly model matrix data. In the new RBM model, both input and hidden variables are in matrix forms which are connected by bilinear transforms. The MVRBM has much less model parameters, resulting in a faster train- ing algorithm while retaining comparable performance as the classic RBM. The advantages of the MVRBM have been demonstrated on two real-world applications: Image super- resolution and handwritten digit recognition.