Closed-form Marginal Likelihood in Gamma-Poisson Matrix Factorization
This provides theoretical insights for researchers in probabilistic matrix factorization, explaining previously observed empirical robustness in rank specification.
The paper tackled the problem of estimating the topic/dictionary matrix in Gamma-Poisson matrix factorization for count data by showing that the model can be rewritten without the activation matrix, leading to maximum marginal likelihood estimation that is robust to over-specified rank and automatically prunes irrelevant columns, as empirically supported.
We present novel understandings of the Gamma-Poisson (GaP) model, a probabilistic matrix factorization model for count data. We show that GaP can be rewritten free of the score/activation matrix. This gives us new insights about the estimation of the topic/dictionary matrix by maximum marginal likelihood estimation. In particular, this explains the robustness of this estimator to over-specified values of the factorization rank, especially its ability to automatically prune irrelevant dictionary columns, as empirically observed in previous work. The marginalization of the activation matrix leads in turn to a new Monte Carlo Expectation-Maximization algorithm with favorable properties.