Estimating a potential without the agony of the partition function
This addresses a computational bottleneck in statistical learning for high-dimensional density estimation, though it appears incremental as it builds on MAP estimators.
The paper tackles the problem of estimating Gibbs density functions without computing the partition function, which is intractable in high dimensions, by proposing a Maximum Recovery MAP (MR-MAP) approach and solving it with a feed-forward hyperbolic neural network, demonstrating effectiveness on standard datasets.
Estimating a Gibbs density function given a sample is an important problem in computational statistics and statistical learning. Although the well established maximum likelihood method is commonly used, it requires the computation of the partition function (i.e., the normalization of the density). This function can be easily calculated for simple low-dimensional problems but its computation is difficult or even intractable for general densities and high-dimensional problems. In this paper we propose an alternative approach based on Maximum A-Posteriori (MAP) estimators, we name Maximum Recovery MAP (MR-MAP), to derive estimators that do not require the computation of the partition function, and reformulate the problem as an optimization problem. We further propose a least-action type potential that allows us to quickly solve the optimization problem as a feed-forward hyperbolic neural network. We demonstrate the effectiveness of our methods on some standard data sets.