Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy
This addresses the latency bottleneck for interactive applications using diffusion models, but it is incremental as it builds on existing denoising techniques.
The paper tackles the high computational cost of step-by-step noise prediction in diffusion models by proposing AdaptiveDiffusion, which adaptively reduces noise prediction steps using a bounded difference approximation strategy, achieving up to 2-5x speedup without quality degradation.
Diffusion models have recently achieved great success in the synthesis of high-quality images and videos. However, the existing denoising techniques in diffusion models are commonly based on step-by-step noise predictions, which suffers from high computation cost, resulting in a prohibitive latency for interactive applications. In this paper, we propose AdaptiveDiffusion to relieve this bottleneck by adaptively reducing the noise prediction steps during the denoising process. Our method considers the potential of skipping as many noise prediction steps as possible while keeping the final denoised results identical to the original full-step ones. Specifically, the skipping strategy is guided by the third-order latent difference that indicates the stability between timesteps during the denoising process, which benefits the reusing of previous noise prediction results. Extensive experiments on image and video diffusion models demonstrate that our method can significantly speed up the denoising process while generating identical results to the original process, achieving up to an average 2~5x speedup without quality degradation.