Bootstrap Your Flow
This addresses a problem for researchers and practitioners in machine learning and computational statistics who need efficient sampling from complex distributions, but it is incremental as it builds on existing flow and AIS techniques.
The paper tackles the challenge of approximating intractable distributions with normalizing flows, which often suffer from mode seeking, high variance, or reliance on target samples, by introducing FAB (Flow AIS Bootstrap), a method combining flows with annealed importance sampling and α-divergence. The result is accurate approximations to complex targets like Boltzmann distributions where previous methods fail.
Normalizing flows are flexible, parameterized distributions that can be used to approximate expectations from intractable distributions via importance sampling. However, current flow-based approaches are limited on challenging targets where they either suffer from mode seeking behaviour or high variance in the training loss, or rely on samples from the target distribution, which may not be available. To address these challenges, we combine flows with annealed importance sampling (AIS), while using the $α$-divergence as our objective, in a novel training procedure, FAB (Flow AIS Bootstrap). Thereby, the flow and AIS improve each other in a bootstrapping manner. We demonstrate that FAB can be used to produce accurate approximations to complex target distributions, including Boltzmann distributions, in problems where previous flow-based methods fail.