Dimension of Marginals of Kronecker Product Models
This work addresses a theoretical problem in statistical modeling and machine learning, specifically for researchers studying exponential families and dimensionality, and is incremental as it builds on existing methods to derive new conditions and proofs.
The paper tackles the problem of estimating the dimension of Kronecker product models, which are exponential families with structured sufficient statistics, by analyzing the Jacobian rank in large parameter limits and using tropical geometry. It provides combinatorial conditions for when these models achieve expected dimension and proves this always holds for binary restricted Boltzmann machines.
A Kronecker product model is the set of visible marginal probability distributions of an exponential family whose sufficient statistics matrix factorizes as a Kronecker product of two matrices, one for the visible variables and one for the hidden variables. We estimate the dimension of these models by the maximum rank of the Jacobian in the limit of large parameters. The limit is described by the tropical morphism; a piecewise linear map with pieces corresponding to slicings of the visible matrix by the normal fan of the hidden matrix. We obtain combinatorial conditions under which the model has the expected dimension, equal to the minimum of the number of natural parameters and the dimension of the ambient probability simplex. Additionally, we prove that the binary restricted Boltzmann machine always has the expected dimension.