Persistently Trained, Diffusion-assisted Energy-based Models
This work addresses a key bottleneck in generative modeling for researchers and practitioners, offering a novel method to improve EBMs for image tasks, though it appears incremental as it builds on existing persistent training and diffusion techniques.
The authors tackled the challenge of training energy-based models (EBMs) for image data, which often struggle with non-convergence in Markov chain Monte Carlo and fail to achieve both image generation and density estimation. They proposed diffusion-assisted EBMs with persistent training, demonstrating for the first time on image data simultaneous long-run stability, post-training image generation, and superior out-of-distribution detection.
Maximum likelihood (ML) learning for energy-based models (EBMs) is challenging, partly due to non-convergence of Markov chain Monte Carlo.Several variations of ML learning have been proposed, but existing methods all fail to achieve both post-training image generation and proper density estimation. We propose to introduce diffusion data and learn a joint EBM, called diffusion assisted-EBMs, through persistent training (i.e., using persistent contrastive divergence) with an enhanced sampling algorithm to properly sample from complex, multimodal distributions. We present results from a 2D illustrative experiment and image experiments and demonstrate that, for the first time for image data, persistently trained EBMs can {\it simultaneously} achieve long-run stability, post-training image generation, and superior out-of-distribution detection.