Variational Inference with Continuously-Indexed Normalizing Flows
This work addresses a bottleneck in variational inference for researchers and practitioners dealing with complex posterior distributions, though it is incremental as it builds on existing CIF methods.
The paper tackled the challenge of using continuously-indexed normalizing flows (CIFs) in variational inference, which lack closed-form densities, by developing an auxiliary VI scheme that leverages CIFs' conditional independence to train expressive posterior approximations. The result showed improved performance over baseline flows in tasks with complex posterior topologies, yielding low-variance evidence estimators and better results in Bayesian inference and surrogate maximum likelihood.
Continuously-indexed flows (CIFs) have recently achieved improvements over baseline normalizing flows on a variety of density estimation tasks. CIFs do not possess a closed-form marginal density, and so, unlike standard flows, cannot be plugged in directly to a variational inference (VI) scheme in order to produce a more expressive family of approximate posteriors. However, we show here how CIFs can be used as part of an auxiliary VI scheme to formulate and train expressive posterior approximations in a natural way. We exploit the conditional independence structure of multi-layer CIFs to build the required auxiliary inference models, which we show empirically yield low-variance estimators of the model evidence. We then demonstrate the advantages of CIFs over baseline flows in VI problems when the posterior distribution of interest possesses a complicated topology, obtaining improved results in both the Bayesian inference and surrogate maximum likelihood settings.