The Dependent Random Measures with Independent Increments in Mixture Models
This work provides a method for improving mixture models in grouped data settings, but it appears incremental as it builds on existing dependent random measures theory.
The paper tackles the problem of modeling grouped observations with shared structure by proposing a framework using normalized dependent random measures with independent increments, deriving the exchangeable probability partition function and an inference algorithm. Experiments with Gaussian mixture models show that the inferred mixing weights and number of clusters respond appropriately to the number of random measures used.
When observations are organized into groups where commonalties exist amongst them, the dependent random measures can be an ideal choice for modeling. One of the propositions of the dependent random measures is that the atoms of the posterior distribution are shared amongst groups, and hence groups can borrow information from each other. When normalized dependent random measures prior with independent increments are applied, we can derive appropriate exchangeable probability partition function (EPPF), and subsequently also deduce its inference algorithm given any mixture model likelihood. We provide all necessary derivation and solution to this framework. For demonstration, we used mixture of Gaussians likelihood in combination with a dependent structure constructed by linear combinations of CRMs. Our experiments show superior performance when using this framework, where the inferred values including the mixing weights and the number of clusters both respond appropriately to the number of completely random measure used.