Transductive Boltzmann Machines
This addresses a fundamental limitation in probabilistic modeling for machine learning, though it appears incremental as it builds on existing Boltzmann machine frameworks.
The paper tackles the problem of learning Gibbs distributions by introducing transductive Boltzmann machines (TBMs), which adaptively construct a minimal sample space from data to avoid combinatorial explosion, and empirically shows TBMs outperform existing Boltzmann machines in efficiency and effectiveness.
We present transductive Boltzmann machines (TBMs), which firstly achieve transductive learning of the Gibbs distribution. While exact learning of the Gibbs distribution is impossible by the family of existing Boltzmann machines due to combinatorial explosion of the sample space, TBMs overcome the problem by adaptively constructing the minimum required sample space from data to avoid unnecessary generalization. We theoretically provide bias-variance decomposition of the KL divergence in TBMs to analyze its learnability, and empirically demonstrate that TBMs are superior to the fully visible Boltzmann machines and popularly used restricted Boltzmann machines in terms of efficiency and effectiveness.