Geometrical Properties of Text Token Embeddings for Strong Semantic Binding in Text-to-Image Generation
This work addresses semantic binding issues in text-to-image models for complex scenes, offering an incremental improvement over existing optimization techniques.
The paper tackles the problem of text-image misalignment in text-to-image generation by investigating the geometrical properties of text token embeddings and cross-attention maps, leading to a training-free framework called TokeBi that outperforms prior methods across diverse baselines and datasets.
Text-to-image (T2I) models often suffer from text-image misalignment in complex scenes involving multiple objects and attributes. Semantic binding has attempted to associate the generated attributes and objects with their corresponding noun phrases (NPs) by text or latent optimizations with the modulation of cross-attention (CA) maps; yet, the factors that influence semantic binding remain underexplored. Here, we investigate the geometrical properties of text token embeddings and their CA maps. We found that the geometrical properties of token embeddings, specifically angular distances and norms, are crucial factors in the differentiation of the CA map. These theoretical findings led to our proposed training-free text-embedding-aware T2I framework, dubbed \textbf{TokeBi}, for strong semantic binding. TokeBi consists of Causality-Aware Projection-Out (CAPO) for distinguishing inter-NP CA maps and Adaptive Token Mixing (ATM) for enhancing inter-NP separation while maintaining intra-NP cohesion in CA maps. Extensive experiments confirm that TokeBi outperforms prior arts across diverse baselines and datasets.