Theory of Dependent Hierarchical Normalized Random Measures
This work provides foundational theory for probabilistic models in machine learning, particularly for applications like topic modeling, but it is incremental as it builds on existing NRM and NGG concepts.
The paper develops a theoretical framework for Dependent Hierarchical Normalized Random Measures (NRMs) and Normalized Generalized Gammas (NGGs), enabling the analysis of networks with dependent and hierarchical structures, such as in time-dependent topic modeling, by introducing dependency operators and composition results.
This paper presents theory for Normalized Random Measures (NRMs), Normalized Generalized Gammas (NGGs), a particular kind of NRM, and Dependent Hierarchical NRMs which allow networks of dependent NRMs to be analysed. These have been used, for instance, for time-dependent topic modelling. In this paper, we first introduce some mathematical background of completely random measures (CRMs) and their construction from Poisson processes, and then introduce NRMs and NGGs. Slice sampling is also introduced for posterior inference. The dependency operators in Poisson processes and for the corresponding CRMs and NRMs is then introduced and Posterior inference for the NGG presented. Finally, we give dependency and composition results when applying these operators to NRMs so they can be used in a network with hierarchical and dependent relations.