Boltzmann Machine Learning with the Latent Maximum Entropy Principle
This work addresses a specific challenge in machine learning for probabilistic models, offering an incremental improvement in robustness and speed for Boltzmann machine training.
The authors tackled the problem of parameter estimation in Boltzmann machines by introducing the latent maximum entropy principle (LME), a new statistical learning paradigm, and demonstrated that LME-based estimation yields better results than maximum likelihood estimation, especially for inferring hidden units with small data.
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from standard maximum likelihood estimation.We demonstrate the LME principle BY deriving new algorithms for Boltzmann machine parameter estimation, and show how robust and fast new variant of the EM algorithm can be developed.Our experiments show that estimation based on LME generally yields better results than maximum likelihood estimation, particularly when inferring hidden units from small amounts of data.