Annealing Flow Generative Models Towards Sampling High-Dimensional and Multi-Modal Distributions
This work addresses the fundamental challenge of sampling from high-dimensional, multi-modal distributions, which is crucial for statistical Bayesian inference and physics-based machine learning.
This paper introduces Annealing Flow (AF), a method based on Continuous Normalizing Flow (CNF) for sampling from high-dimensional, multi-modal distributions. AF uses a dynamic Optimal Transport objective with Wasserstein regularization and annealing procedures to explore modes effectively, demonstrating superior performance over state-of-the-art methods in various challenging distributions and real-world datasets.
Sampling from high-dimensional, multi-modal distributions remains a fundamental challenge across domains such as statistical Bayesian inference and physics-based machine learning. In this paper, we propose Annealing Flow (AF), a method built on Continuous Normalizing Flow (CNF) for sampling from high-dimensional and multi-modal distributions. AF is trained with a dynamic Optimal Transport (OT) objective incorporating Wasserstein regularization, and guided by annealing procedures, facilitating effective exploration of modes in high-dimensional spaces. Compared to recent NF methods, AF greatly improves training efficiency and stability, with minimal reliance on MC assistance. We demonstrate the superior performance of AF compared to state-of-the-art methods through experiments on various challenging distributions and real-world datasets, particularly in high-dimensional and multi-modal settings. We also highlight AF potential for sampling the least favorable distributions.