Balanced Training of Energy-Based Models with Adaptive Flow Sampling
This work addresses a specific bottleneck in EBM training for researchers in generative modeling, though it appears incremental as it builds on existing sampling techniques.
The authors tackled the problem of training energy-based models (EBMs) with intractable likelihoods by proposing a new maximum likelihood algorithm that uses normalizing flows for adaptive sampling, resulting in a fast sampler for generating new data.
Energy-based models (EBMs) are versatile density estimation models that directly parameterize an unnormalized log density. Although very flexible, EBMs lack a specified normalization constant of the model, making the likelihood of the model computationally intractable. Several approximate samplers and variational inference techniques have been proposed to estimate the likelihood gradients for training. These techniques have shown promising results in generating samples, but little attention has been paid to the statistical accuracy of the estimated density, such as determining the relative importance of different classes in a dataset. In this work, we propose a new maximum likelihood training algorithm for EBMs that uses a different type of generative model, normalizing flows (NF), which have recently been proposed to facilitate sampling. Our method fits an NF to an EBM during training so that an NF-assisted sampling scheme provides an accurate gradient for the EBMs at all times, ultimately leading to a fast sampler for generating new data.