PAVI: Plate-Amortized Variational Inference
This work addresses scalability issues in variational inference for researchers dealing with large datasets like neuroimaging, though it is incremental as it builds on existing VI methods with a novel parameterization approach.
The paper tackles the computational challenge of Bayesian inference in large population studies with massive latent parameter spaces by introducing plate-amortized variational inference (PAVI), which shares parameterization across i.i.d. variables to create expressive, parsimoniously parameterized variational families. In a neuroimaging example with a million latent parameters, PAVI achieves orders of magnitude faster training while maintaining expressiveness.
Given some observed data and a probabilistic generative model, Bayesian inference aims at obtaining the distribution of a model's latent parameters that could have yielded the data. This task is challenging for large population studies where thousands of measurements are performed over a cohort of hundreds of subjects, resulting in a massive latent parameter space. This large cardinality renders off-the-shelf Variational Inference (VI) computationally impractical. In this work, we design structured VI families that can efficiently tackle large population studies. To this end, our main idea is to share the parameterization and learning across the different i.i.d. variables in a generative model -symbolized by the model's plates. We name this concept plate amortization, and illustrate the powerful synergies it entitles, resulting in expressive, parsimoniously parameterized and orders of magnitude faster to train large scale hierarchical variational distributions. We illustrate the practical utility of PAVI through a challenging Neuroimaging example featuring a million latent parameters, demonstrating a significant step towards scalable and expressive Variational Inference.