Auxiliary Variables for Multi-Dirichlet Priors
This work addresses a technical bottleneck in Bayesian modeling for researchers using hierarchical Dirichlet priors, though it appears incremental in scope.
The paper tackles the challenge of inferring mixing weights and parameters in Bayesian models with Multi-Dirichlet priors by introducing a novel auxiliary variable scheme, which simplifies inference and enables efficient, fully collapsed inference methods.
Bayesian models that mix multiple Dirichlet prior parameters, called Multi-Dirichlet priors (MD) in this paper, are gaining popularity. Inferring mixing weights and parameters of mixed prior distributions seems tricky, as sums over Dirichlet parameters complicate the joint distribution of model parameters. This paper shows a novel auxiliary variable scheme which helps to simplify the inference for models involving hierarchical MDs and MDPs. Using this scheme, it is easy to derive fully collapsed inference schemes which allow for an efficient inference.