Post-training Quantization for Text-to-Image Diffusion Models with Progressive Calibration and Activation Relaxing
This work addresses the computational inefficiency of widely-used pretrained text-to-image diffusion models, offering a practical compression solution with a new benchmark for accurate evaluation.
The paper tackles the problem of high computational overhead in text-to-image diffusion models like Stable Diffusion by proposing a post-training quantization method called PCR, which achieves quantization for Stable Diffusion XL while maintaining performance, as demonstrated through extensive experiments.
High computational overhead is a troublesome problem for diffusion models. Recent studies have leveraged post-training quantization (PTQ) to compress diffusion models. However, most of them only focus on unconditional models, leaving the quantization of widely-used pretrained text-to-image models, e.g., Stable Diffusion, largely unexplored. In this paper, we propose a novel post-training quantization method PCR (Progressive Calibration and Relaxing) for text-to-image diffusion models, which consists of a progressive calibration strategy that considers the accumulated quantization error across timesteps, and an activation relaxing strategy that improves the performance with negligible cost. Additionally, we demonstrate the previous metrics for text-to-image diffusion model quantization are not accurate due to the distribution gap. To tackle the problem, we propose a novel QDiffBench benchmark, which utilizes data in the same domain for more accurate evaluation. Besides, QDiffBench also considers the generalization performance of the quantized model outside the calibration dataset. Extensive experiments on Stable Diffusion and Stable Diffusion XL demonstrate the superiority of our method and benchmark. Moreover, we are the first to achieve quantization for Stable Diffusion XL while maintaining the performance.