Breaking the Lock-in: Diversifying Text-to-Image Generation via Representation Modulation
For users of text-to-image models, this work provides a lightweight solution to the practical problem of low sample diversity without sacrificing quality or speed.
The paper identifies that early convergence of the DC component in Transformer features causes low diversity in text-to-image generation. DAVE, a training-free method that attenuates this component, improves diversity by up to 30% while maintaining image quality and requiring negligible overhead.
Recent text-to-image models built on large-scale Transformer backbones and flow-based objectives deliver strong text-image alignment and high visual quality, yet often produce overly similar samples under a fixed prompt. Existing diversity-enhancement methods alleviate this issue, but typically require expensive sampling or auxiliary optimization, incurring non-trivial overhead. To investigate the root cause of this homogeneity, we examine intermediate Transformer features and observe that the zero-frequency spatial average (DC) component rapidly converges across seeds early in generation, causing early trajectory lock-in that limits downstream variation. Building on this observation, we propose DC Attenuation for diVersity Enhancement (DAVE), a training-free representation-level intervention that selectively attenuates this component in the early regime. DAVE preserves the sampling pipeline with negligible overhead, improving prompt-consistent diversity while maintaining competitive image quality.