Nested Diffusion Processes for Anytime Image Generation
This work addresses the slow generation speed of diffusion models, which is a bottleneck for real-time applications like content creation and inverse problem solving, though it is incremental as it builds on existing pretrained models.
The paper tackles the computational expense of diffusion models for image generation by proposing a method that enables anytime generation with intermediate images of higher quality than the original model, while maintaining comparable final results, as demonstrated on ImageNet and Stable Diffusion with qualitative and quantitative improvements.
Diffusion models are the current state-of-the-art in image generation, synthesizing high-quality images by breaking down the generation process into many fine-grained denoising steps. Despite their good performance, diffusion models are computationally expensive, requiring many neural function evaluations (NFEs). In this work, we propose an anytime diffusion-based method that can generate viable images when stopped at arbitrary times before completion. Using existing pretrained diffusion models, we show that the generation scheme can be recomposed as two nested diffusion processes, enabling fast iterative refinement of a generated image. In experiments on ImageNet and Stable Diffusion-based text-to-image generation, we show, both qualitatively and quantitatively, that our method's intermediate generation quality greatly exceeds that of the original diffusion model, while the final generation result remains comparable. We illustrate the applicability of Nested Diffusion in several settings, including for solving inverse problems, and for rapid text-based content creation by allowing user intervention throughout the sampling process.