Iterative Importance Fine-tuning of Diffusion Models
This work addresses a key bottleneck in applying diffusion models to tasks like imaging and protein design, offering an incremental improvement for researchers and practitioners in generative modeling.
The paper tackles the challenge of efficiently sampling from posterior distributions in diffusion models for downstream tasks by introducing a self-supervised algorithm that iteratively refines the h-transform using synthetic data with importance weights, demonstrating effectiveness in class-conditional sampling, inverse problems, and reward fine-tuning for text-to-image models.
Diffusion models are an important tool for generative modelling, serving as effective priors in applications such as imaging and protein design. A key challenge in applying diffusion models for downstream tasks is efficiently sampling from resulting posterior distributions, which can be addressed using the $h$-transform. This work introduces a self-supervised algorithm for fine-tuning diffusion models by estimating the $h$-transform, enabling amortised conditional sampling. Our method iteratively refines the $h$-transform using a synthetic dataset resampled with path-based importance weights. We demonstrate the effectiveness of this framework on class-conditional sampling, inverse problems and reward fine-tuning for text-to-image diffusion models.