SCORENF: Score-based Normalizing Flows for Sampling Unnormalized distributions
This work addresses sampling inefficiencies for researchers in scientific domains like physics, though it appears incremental as it builds on existing normalizing flow and score-based methods.
The paper tackled the problem of sampling from unnormalized probability distributions, which is challenging due to slow convergence and mode issues in traditional methods, and proposed ScoreNF, a score-based normalizing flow framework that enables efficient and unbiased sampling, validated on synthetic and high-dimensional distributions.
Unnormalized probability distributions are central to modeling complex physical systems across various scientific domains. Traditional sampling methods, such as Markov Chain Monte Carlo (MCMC), often suffer from slow convergence, critical slowing down, poor mode mixing, and high autocorrelation. In contrast, likelihood-based and adversarial machine learning models, though effective, are heavily data-driven, requiring large datasets and often encountering mode covering and mode collapse. In this work, we propose ScoreNF, a score-based learning framework built on the Normalizing Flow (NF) architecture, integrated with an Independent Metropolis-Hastings (IMH) module, enabling efficient and unbiased sampling from unnormalized target distributions. We show that ScoreNF maintains high performance even with small training ensembles, thereby reducing reliance on computationally expensive MCMC-generated training data. We also present a method for assessing mode-covering and mode-collapse behaviours. We validate our method on synthetic 2D distributions (MOG-4 and MOG-8) and the high-dimensional $φ^4$ lattice field theory distribution, demonstrating its effectiveness for sampling tasks.