Bayesian inference in hierarchical models by combining independent posteriors
This work addresses computational bottlenecks for researchers and practitioners using hierarchical models in statistics and machine learning, though it is incremental as it builds on existing inference techniques.
The paper tackles the computational challenge of fully Bayesian inference in hierarchical models with complex or numerous data sources by proposing an approach that first performs independent inference for each source and then uses the posterior samples in a substitute hierarchical model. This method speeds up convergence and enables parallel processing, as demonstrated with simulated and real data.
Hierarchical models are versatile tools for joint modeling of data sets arising from different, but related, sources. Fully Bayesian inference may, however, become computationally prohibitive if the source-specific data models are complex, or if the number of sources is very large. To facilitate computation, we propose an approach, where inference is first made independently for the parameters of each data set, whereupon the obtained posterior samples are used as observed data in a substitute hierarchical model, based on a scaled likelihood function. Compared to direct inference in a full hierarchical model, the approach has the advantage of being able to speed up convergence by breaking down the initial large inference problem into smaller individual subproblems with better convergence properties. Moreover it enables parallel processing of the possibly complex inferences of the source-specific parameters, which may otherwise create a computational bottleneck if processed jointly as part of a hierarchical model. The approach is illustrated with both simulated and real data.