AutoBayes: A Compositional Framework for Generalized Variational Inference
This work addresses the challenge of designing and optimizing variational inference models for researchers in machine learning and statistics, though it appears incremental as it builds on existing concepts like chain rules and compositional methods.
The paper tackles the complexity of building and optimizing generalized variational inference models by introducing a compositional framework that clarifies model parts and their interactions, resulting in tools for model construction, inversion, and local optimization.
We introduce a new compositional framework for generalized variational inference, clarifying the different parts of a model, how they interact, and how they compose. We explain that both exact Bayesian inference and the loss functions typical of variational inference (such as variational free energy and its generalizations) satisfy chain rules akin to that of reverse-mode automatic differentiation, and we advocate for exploiting this to build and optimize models accordingly. To this end, we construct a series of compositional tools: for building models; for constructing their inversions; for attaching local loss functions; and for exposing parameters. Finally, we explain how the resulting parameterized statistical games may be optimized locally, too. We illustrate our framework with a number of classic examples, pointing to new areas of extensibility that are revealed.