Provable Statistical Rates for Consistency Diffusion Models
It provides theoretical foundations for consistency models, addressing a gap in understanding for researchers and practitioners in generative modeling.
The paper tackles the lack of statistical theory for consistency models, which speed up diffusion models, by formulating training as distribution discrepancy minimization and deriving Wasserstein distance estimation rates that match those of vanilla diffusion models.
Diffusion models have revolutionized various application domains, including computer vision and audio generation. Despite the state-of-the-art performance, diffusion models are known for their slow sample generation due to the extensive number of steps involved. In response, consistency models have been developed to merge multiple steps in the sampling process, thereby significantly boosting the speed of sample generation without compromising quality. This paper contributes towards the first statistical theory for consistency models, formulating their training as a distribution discrepancy minimization problem. Our analysis yields statistical estimation rates based on the Wasserstein distance for consistency models, matching those of vanilla diffusion models. Additionally, our results encompass the training of consistency models through both distillation and isolation methods, demystifying their underlying advantage.