ShiftDDPMs: Exploring Conditional Diffusion Models by Shifting Diffusion Trajectories
This addresses a problem in conditional image generation for researchers and practitioners by proposing a novel approach to improve modeling flexibility, though it appears incremental as it builds on existing diffusion models.
The paper tackles the limitation of conditional diffusion models by introducing conditions into the forward process, using extra latent space to allocate exclusive diffusion trajectories for each condition, which improves condition modeling across all timesteps and enhances learning capacity, as demonstrated through experiments on image synthesis.
Diffusion models have recently exhibited remarkable abilities to synthesize striking image samples since the introduction of denoising diffusion probabilistic models (DDPMs). Their key idea is to disrupt images into noise through a fixed forward process and learn its reverse process to generate samples from noise in a denoising way. For conditional DDPMs, most existing practices relate conditions only to the reverse process and fit it to the reversal of unconditional forward process. We find this will limit the condition modeling and generation in a small time window. In this paper, we propose a novel and flexible conditional diffusion model by introducing conditions into the forward process. We utilize extra latent space to allocate an exclusive diffusion trajectory for each condition based on some shifting rules, which will disperse condition modeling to all timesteps and improve the learning capacity of model. We formulate our method, which we call \textbf{ShiftDDPMs}, and provide a unified point of view on existing related methods. Extensive qualitative and quantitative experiments on image synthesis demonstrate the feasibility and effectiveness of ShiftDDPMs.