Denoising Diffusion Step-aware Models
This addresses efficiency bottlenecks for researchers and practitioners using diffusion models in image generation, though it is incremental as it builds on existing methods.
The paper tackled the high computational overhead in Denoising Diffusion Probabilistic Models (DDPMs) by introducing Denoising Diffusion Step-aware Models (DDSM), which uses step-adapted neural networks to achieve computational savings of 49% to 76% across datasets like CIFAR-10 and ImageNet without quality loss.
Denoising Diffusion Probabilistic Models (DDPMs) have garnered popularity for data generation across various domains. However, a significant bottleneck is the necessity for whole-network computation during every step of the generative process, leading to high computational overheads. This paper presents a novel framework, Denoising Diffusion Step-aware Models (DDSM), to address this challenge. Unlike conventional approaches, DDSM employs a spectrum of neural networks whose sizes are adapted according to the importance of each generative step, as determined through evolutionary search. This step-wise network variation effectively circumvents redundant computational efforts, particularly in less critical steps, thereby enhancing the efficiency of the diffusion model. Furthermore, the step-aware design can be seamlessly integrated with other efficiency-geared diffusion models such as DDIMs and latent diffusion, thus broadening the scope of computational savings. Empirical evaluations demonstrate that DDSM achieves computational savings of 49% for CIFAR-10, 61% for CelebA-HQ, 59% for LSUN-bedroom, 71% for AFHQ, and 76% for ImageNet, all without compromising the generation quality.