Quantizing Diffusion Models from a Sampling-Aware Perspective
This work addresses efficiency challenges in diffusion models for resource-limited environments, representing an incremental improvement by integrating quantization with sampling-aware constraints.
The paper tackles the problem of accelerating diffusion models for low-latency applications by addressing how quantization-induced noise disrupts sampling trajectories, proposing a sampling-aware quantization strategy that maintains high fidelity and rapid convergence in experiments across multiple datasets.
Diffusion models have recently emerged as the dominant approach in visual generation tasks. However, the lengthy denoising chains and the computationally intensive noise estimation networks hinder their applicability in low-latency and resource-limited environments. Previous research has endeavored to address these limitations in a decoupled manner, utilizing either advanced samplers or efficient model quantization techniques. In this study, we uncover that quantization-induced noise disrupts directional estimation at each sampling step, further distorting the precise directional estimations of higher-order samplers when solving the sampling equations through discretized numerical methods, thereby altering the optimal sampling trajectory. To attain dual acceleration with high fidelity, we propose a sampling-aware quantization strategy, wherein a Mixed-Order Trajectory Alignment technique is devised to impose a more stringent constraint on the error bounds at each sampling step, facilitating a more linear probability flow. Extensive experiments on sparse-step fast sampling across multiple datasets demonstrate that our approach preserves the rapid convergence characteristics of high-speed samplers while maintaining superior generation quality. Code will be made publicly available soon.