SoftCFG: Uncertainty-guided Stable Guidance for Visual Autoregressive Model
This addresses stability and quality problems in conditional image generation for AI and computer vision applications, representing an incremental improvement over existing methods.
The paper tackled the issues of guidance diminishing and over-guidance in applying Classifier-Free Guidance to autoregressive image generation models, resulting in SoftCFG, which improved image quality and achieved state-of-the-art FID on ImageNet 256*256 among autoregressive models.
Autoregressive (AR) models have emerged as powerful tools for image generation by modeling images as sequences of discrete tokens. While Classifier-Free Guidance (CFG) has been adopted to improve conditional generation, its application in AR models faces two key issues: guidance diminishing, where the conditional-unconditional gap quickly vanishes as decoding progresses, and over-guidance, where strong conditions distort visual coherence. To address these challenges, we propose SoftCFG, an uncertainty-guided inference method that distributes adaptive perturbations across all tokens in the sequence. The key idea behind SoftCFG is to let each generated token contribute certainty-weighted guidance, ensuring that the signal persists across steps while resolving conflicts between text guidance and visual context. To further stabilize long-sequence generation, we introduce Step Normalization, which bounds cumulative perturbations of SoftCFG. Our method is training-free, model-agnostic, and seamlessly integrates with existing AR pipelines. Experiments show that SoftCFG significantly improves image quality over standard CFG and achieves state-of-the-art FID on ImageNet 256*256 among autoregressive models.