Borrowing strengh in hierarchical Bayes: Posterior concentration of the Dirichlet base measure
This work provides theoretical justification for the efficiency gains in hierarchical Bayesian modeling, particularly for statisticians and machine learning researchers dealing with multi-group data, though it is incremental in extending existing convergence theory.
The paper analyzes the posterior concentration behavior of the base measure in hierarchical Dirichlet processes as observations increase, establishing convergence rates in Wasserstein metrics under various geometric conditions. It demonstrates that hierarchical modeling can dramatically improve efficiency, shifting from nonparametric to parametric convergence rates in certain settings.
This paper studies posterior concentration behavior of the base probability measure of a Dirichlet measure, given observations associated with the sampled Dirichlet processes, as the number of observations tends to infinity. The base measure itself is endowed with another Dirichlet prior, a construction known as the hierarchical Dirichlet processes (Teh et al. [J. Amer. Statist. Assoc. 101 (2006) 1566-1581]). Convergence rates are established in transportation distances (i.e., Wasserstein metrics) under various conditions on the geometry of the support of the true base measure. As a consequence of the theory, we demonstrate the benefit of "borrowing strength" in the inference of multiple groups of data - a powerful insight often invoked to motivate hierarchical modeling. In certain settings, the gain in efficiency due to the latent hierarchy can be dramatic, improving from a standard nonparametric rate to a parametric rate of convergence. Tools developed include transportation distances for nonparametric Bayesian hierarchies of random measures, the existence of tests for Dirichlet measures, and geometric properties of the support of Dirichlet measures.