SDDM: Score-Decomposed Diffusion Models on Manifolds for Unpaired Image-to-Image Translation
This work addresses a bottleneck in diffusion models for image translation, offering an incremental improvement in efficiency and performance for computer vision applications.
The paper tackles the problem of unpaired image-to-image translation by proposing SDDM, a score-decomposed diffusion model on manifolds that explicitly optimizes intermediate generative distributions, resulting in outperforming existing methods with fewer diffusion steps on benchmarks.
Recent score-based diffusion models (SBDMs) show promising results in unpaired image-to-image translation (I2I). However, existing methods, either energy-based or statistically-based, provide no explicit form of the interfered intermediate generative distributions. This work presents a new score-decomposed diffusion model (SDDM) on manifolds to explicitly optimize the tangled distributions during image generation. SDDM derives manifolds to make the distributions of adjacent time steps separable and decompose the score function or energy guidance into an image ``denoising" part and a content ``refinement" part. To refine the image in the same noise level, we equalize the refinement parts of the score function and energy guidance, which permits multi-objective optimization on the manifold. We also leverage the block adaptive instance normalization module to construct manifolds with lower dimensions but still concentrated with the perturbed reference image. SDDM outperforms existing SBDM-based methods with much fewer diffusion steps on several I2I benchmarks.