Spectrally-Guided Diffusion Noise Schedules
This work addresses a specific bottleneck in diffusion models for image generation, offering an incremental improvement over existing methods.
The paper tackles the problem of manually tuned noise schedules in diffusion models by proposing a principled method to design per-instance schedules based on image spectral properties, resulting in improved generative quality, especially with fewer steps.
Denoising diffusion models are widely used for high-quality image and video generation. Their performance depends on noise schedules, which define the distribution of noise levels applied during training and the sequence of noise levels traversed during sampling. Noise schedules are typically handcrafted and require manual tuning across different resolutions. In this work, we propose a principled way to design per-instance noise schedules for pixel diffusion, based on the image's spectral properties. By deriving theoretical bounds on the efficacy of minimum and maximum noise levels, we design ``tight'' noise schedules that eliminate redundant steps. During inference, we propose to conditionally sample such noise schedules. Experiments show that our noise schedules improve generative quality of single-stage pixel diffusion models, particularly in the low-step regime.