CREPE: Controlling Diffusion with Replica Exchange
This work addresses the challenge of steering diffusion model outputs to meet new constraints without retraining, offering an incremental improvement over prior Sequential Monte Carlo methods.
The paper tackles the problem of inference-time control for diffusion models by proposing CREPE, a flexible method based on replica exchange, which generates diverse samples and allows online refinement, achieving competitive performance across tasks like temperature annealing and classifier-free guidance debiasing.
Inference-time control of diffusion models aims to steer model outputs to satisfy new constraints without retraining. Previous approaches have mostly relied on heuristic guidance or have been coupled with Sequential Monte Carlo (SMC) for bias correction. In this paper, we propose a flexible alternative based on replica exchange, an algorithm designed initially for sampling problems. We refer to this method as the CREPE (Controlling with REPlica Exchange). Unlike SMC, CREPE: (1) generates particles sequentially, (2) maintains high diversity in the generated samples after a burn-in period, and (3) enables online refinement or early termination. We demonstrate its versatility across various tasks, including temperature annealing, reward-tilting, model composition and classifier-free guidance debiasing, with competitive performance compared to prior SMC methods.