S$^2$-Guidance: Stochastic Self Guidance for Training-Free Enhancement of Diffusion Models
This addresses quality issues in diffusion models for generative AI applications, representing an incremental improvement over existing guidance techniques.
The paper tackled the problem of suboptimal results and semantic incoherence in diffusion models using Classifier-free Guidance (CFG) by proposing S^2-Guidance, a method that uses stochastic sub-networks to refine predictions, resulting in superior performance on text-to-image and text-to-video generation tasks compared to CFG and other strategies.
Classifier-free Guidance (CFG) is a widely used technique in modern diffusion models for enhancing sample quality and prompt adherence. However, through an empirical analysis on Gaussian mixture modeling with a closed-form solution, we observe a discrepancy between the suboptimal results produced by CFG and the ground truth. The model's excessive reliance on these suboptimal predictions often leads to semantic incoherence and low-quality outputs. To address this issue, we first empirically demonstrate that the model's suboptimal predictions can be effectively refined using sub-networks of the model itself. Building on this insight, we propose S^2-Guidance, a novel method that leverages stochastic block-dropping during the forward process to construct stochastic sub-networks, effectively guiding the model away from potential low-quality predictions and toward high-quality outputs. Extensive qualitative and quantitative experiments on text-to-image and text-to-video generation tasks demonstrate that S^2-Guidance delivers superior performance, consistently surpassing CFG and other advanced guidance strategies. Our code will be released.