Multistep Consistency Models
This addresses the problem of balancing efficiency and sample quality in generative AI for applications like image synthesis, though it is incremental as it builds on existing consistency and diffusion models.
The paper tackles the trade-off between sampling speed and quality in generative models by proposing Multistep Consistency Models, which unify consistency models and diffusion models to allow interpolation between them, achieving results like 1.4 FID on Imagenet 64 in 8 steps and 2.1 FID on Imagenet 128 in 8 steps.
Diffusion models are relatively easy to train but require many steps to generate samples. Consistency models are far more difficult to train, but generate samples in a single step. In this paper we propose Multistep Consistency Models: A unification between Consistency Models (Song et al., 2023) and TRACT (Berthelot et al., 2023) that can interpolate between a consistency model and a diffusion model: a trade-off between sampling speed and sampling quality. Specifically, a 1-step consistency model is a conventional consistency model whereas a $\infty$-step consistency model is a diffusion model. Multistep Consistency Models work really well in practice. By increasing the sample budget from a single step to 2-8 steps, we can train models more easily that generate higher quality samples, while retaining much of the sampling speed benefits. Notable results are 1.4 FID on Imagenet 64 in 8 step and 2.1 FID on Imagenet128 in 8 steps with consistency distillation, using simple losses without adversarial training. We also show that our method scales to a text-to-image diffusion model, generating samples that are close to the quality of the original model.