Bridging Diffusion Guidance and Anderson Acceleration via Hopfield Dynamics
This work addresses the problem of efficient and effective diffusion model guidance for the machine learning community, particularly those working with generative models, and offers an incremental yet impactful solution.
This work tackles the problem of improving the generative quality of diffusion models, achieving significant improvements through the proposed Geometry Aware Attention Guidance (GAG) method. The method stabilizes the acceleration process and maximizes guidance efficiency, though no concrete numbers are provided.
Classifier-Free Guidance (CFG) has significantly enhanced the generative quality of diffusion models by extrapolating between conditional and unconditional outputs. However, its high inference cost and limited applicability to distilled or single-step models have shifted research focus toward attention-space extrapolation. While these methods offer computational efficiency, their theoretical underpinnings remain elusive. In this work, we establish a foundational framework for attention-space extrapolation by modeling attention dynamics as fixed-point iterations within Modern Hopfield Networks. We demonstrate that the extrapolation effect in attention space constitutes a special case of Anderson Acceleration applied to these dynamics. Building on this insight and the weak contraction property, we propose Geometry Aware Attention Guidance (GAG). By decomposing attention updates into parallel and orthogonal components relative to the guidance direction, GAG stabilizes the acceleration process and maximizes guidance efficiency. Our plug-and-play method seamlessly integrates with existing frameworks while significantly improving generation quality.