Matrix Product Operator Restricted Boltzmann Machines
This work addresses a domain-specific problem for researchers in machine learning and tensor data analysis, offering an incremental improvement over existing matrix-variate and tensor-variate RBMs.
The paper tackles the problem of conventional restricted Boltzmann machines (RBMs) losing structural information in tensor data by proposing a matrix product operator RBM (MPORBM) that preserves input formats, resulting in higher expressive power for tasks like classification and image completion, with numerical experiments showing improved performance over traditional RBMs.
A restricted Boltzmann machine (RBM) learns a probability distribution over its input samples and has numerous uses like dimensionality reduction, classification and generative modeling. Conventional RBMs accept vectorized data that dismisses potentially important structural information in the original tensor (multi-way) input. Matrix-variate and tensor-variate RBMs, named MvRBM and TvRBM, have been proposed but are all restrictive by model construction, which leads to a weak model expression power. This work presents the matrix product operator RBM (MPORBM) that utilizes a tensor network generalization of Mv/TvRBM, preserves input formats in both the visible and hidden layers, and results in higher expressive power. A novel training algorithm integrating contrastive divergence and an alternating optimization procedure is also developed. Numerical experiments compare the MPORBM with the traditional RBM and MvRBM for data classification and image completion and denoising tasks. The expressive power of the MPORBM as a function of the MPO-rank is also investigated.