Optimizing DDPM Sampling with Shortcut Fine-Tuning
This addresses the challenge of efficient sampling in diffusion models for AI practitioners, though it appears incremental as it builds on existing fast DDPM samplers.
The study tackled the problem of fast sampling in pretrained Denoising Diffusion Probabilistic Models (DDPMs) by proposing Shortcut Fine-Tuning (SFT), which fine-tunes samplers to minimize Integral Probability Metrics, resulting in sample quality comparable to or surpassing the full-step model across various datasets.
In this study, we propose Shortcut Fine-Tuning (SFT), a new approach for addressing the challenge of fast sampling of pretrained Denoising Diffusion Probabilistic Models (DDPMs). SFT advocates for the fine-tuning of DDPM samplers through the direct minimization of Integral Probability Metrics (IPM), instead of learning the backward diffusion process. This enables samplers to discover an alternative and more efficient sampling shortcut, deviating from the backward diffusion process. Inspired by a control perspective, we propose a new algorithm SFT-PG: Shortcut Fine-Tuning with Policy Gradient, and prove that under certain assumptions, gradient descent of diffusion models with respect to IPM is equivalent to performing policy gradient. To our best knowledge, this is the first attempt to utilize reinforcement learning (RL) methods to train diffusion models. Through empirical evaluation, we demonstrate that our fine-tuning method can further enhance existing fast DDPM samplers, resulting in sample quality comparable to or even surpassing that of the full-step model across various datasets.