Discrete Restricted Boltzmann Machines
This work addresses the theoretical understanding of discrete variants of restricted Boltzmann machines for researchers in machine learning and statistics, but it appears incremental as it extends existing binary models to more general discrete cases.
The paper introduces discrete restricted Boltzmann machines as probabilistic graphical models with bipartite interactions between discrete visible and hidden variables, and provides bounds on the number of hidden variables needed to approximate any probability distribution on visible states to a given accuracy.
We describe discrete restricted Boltzmann machines: probabilistic graphical models with bipartite interactions between visible and hidden discrete variables. Examples are binary restricted Boltzmann machines and discrete naive Bayes models. We detail the inference functions and distributed representations arising in these models in terms of configurations of projected products of simplices and normal fans of products of simplices. We bound the number of hidden variables, depending on the cardinalities of their state spaces, for which these models can approximate any probability distribution on their visible states to any given accuracy. In addition, we use algebraic methods and coding theory to compute their dimension.