Accelerating Diffusion-based Super-Resolution with Dynamic Time-Spatial Sampling
This work addresses the computational inefficiency in super-resolution tasks for applications requiring high-quality image generation, though it is incremental as it builds on existing diffusion models.
The paper tackles the high computational cost of diffusion-based super-resolution methods by analyzing frequency- and spatial-domain properties, proposing a Time-Spatial-aware Sampling strategy (TSS) that achieves state-of-the-art performance with significantly fewer iterations, improving MUSIQ scores by 0.2-3.0 and outperforming current acceleration methods with half the steps.
Diffusion models have gained attention for their success in modeling complex distributions, achieving impressive perceptual quality in SR tasks. However, existing diffusion-based SR methods often suffer from high computational costs, requiring numerous iterative steps for training and inference. Existing acceleration techniques, such as distillation and solver optimization, are generally task-agnostic and do not fully leverage the specific characteristics of low-level tasks like super-resolution (SR). In this study, we analyze the frequency- and spatial-domain properties of diffusion-based SR methods, revealing key insights into the temporal and spatial dependencies of high-frequency signal recovery. Specifically, high-frequency details benefit from concentrated optimization during early and late diffusion iterations, while spatially textured regions demand adaptive denoising strategies. Building on these observations, we propose the Time-Spatial-aware Sampling strategy (TSS) for the acceleration of Diffusion SR without any extra training cost. TSS combines Time Dynamic Sampling (TDS), which allocates more iterations to refining textures, and Spatial Dynamic Sampling (SDS), which dynamically adjusts strategies based on image content. Extensive evaluations across multiple benchmarks demonstrate that TSS achieves state-of-the-art (SOTA) performance with significantly fewer iterations, improving MUSIQ scores by 0.2 - 3.0 and outperforming the current acceleration methods with only half the number of steps.