Beyond and Free from Diffusion: Invertible Guided Consistency Training
This work addresses the problem of efficient and high-quality guided image generation for computer vision applications, providing an incremental yet significant improvement over existing methods.
The authors tackled the problem of guided image generation in Consistency Models, achieving a precision of 0.8 at a guidance of 13, outperforming Diffusion Models which dropped to 0.47. This was done through their proposed invertible Guided Consistency Training framework.
Guidance in image generation steers models towards higher-quality or more targeted outputs, typically achieved in Diffusion Models (DMs) via Classifier-free Guidance (CFG). However, recent Consistency Models (CMs), which offer fewer function evaluations, rely on distilling CFG knowledge from pretrained DMs to achieve guidance, making them costly and inflexible. In this work, we propose invertible Guided Consistency Training (iGCT), a novel training framework for guided CMs that is entirely data-driven. iGCT, as a pioneering work, contributes to fast and guided image generation and editing without requiring the training and distillation of DMs, greatly reducing the overall compute requirements. iGCT addresses the saturation artifacts seen in CFG under high guidance scales. Our extensive experiments on CIFAR-10 and ImageNet64 show that iGCT significantly improves FID and precision compared to CFG. At a guidance of 13, iGCT improves precision to 0.8, while DM's drops to 0.47. Our work takes the first step toward enabling guidance and inversion for CMs without relying on DMs.