Bayesian Inference for Gamma Models
This provides a scalable solution for Bayesian inference in various statistical and machine learning models with gamma functions, though it appears incremental as an extension of existing data augmentation techniques.
The paper tackles the challenge of Bayesian inference for models containing gamma functions, which lack natural conjugate priors, by developing a data augmentation scheme using Exponential Reciprocal Gamma distributions. This enables scalable EM and MCMC algorithms, as demonstrated in examples like gamma shape inference and negative binomial regression.
We use the theory of normal variance-mean mixtures to derive a data augmentation scheme for models that include gamma functions. Our methodology applies to many situations in statistics and machine learning, including Multinomial-Dirichlet distributions, Negative binomial regression, Poisson-Gamma hierarchical models, Extreme value models, to name but a few. All of those models include a gamma function which does not admit a natural conjugate prior distribution providing a significant challenge to inference and prediction. To provide a data augmentation strategy, we construct and develop the theory of the class of Exponential Reciprocal Gamma distributions. This allows scalable EM and MCMC algorithms to be developed. We illustrate our methodology on a number of examples, including gamma shape inference, negative binomial regression and Dirichlet allocation. Finally, we conclude with directions for future research.