Selective Guidance: Are All the Denoising Steps of Guided Diffusion Important?
This incremental improvement addresses speed bottlenecks for users of diffusion models in image generation.
The study tackled the computational inefficiency of Stable Diffusion's guided inference pipeline by optimizing denoising steps, reducing complexity by 50% and achieving up to 20.3% faster inference time with minimal visual quality loss.
This study examines the impact of optimizing the Stable Diffusion (SD) guided inference pipeline. We propose optimizing certain denoising steps by limiting the noise computation to conditional noise and eliminating unconditional noise computation, thereby reducing the complexity of the target iterations by 50%. Additionally, we demonstrate that later iterations of the SD are less sensitive to optimization, making them ideal candidates for applying the suggested optimization. Our experiments show that optimizing the last 20% of the denoising loop iterations results in an 8.2% reduction in inference time with almost no perceivable changes to the human eye. Furthermore, we found that by extending the optimization to 50% of the last iterations, we can reduce inference time by approximately 20.3%, while still generating visually pleasing images.