TextGuider: Training-Free Guidance for Text Rendering via Attention Alignment
This addresses text rendering issues in AI-generated images, which is a domain-specific problem for users needing accurate text in images, though it is incremental as it builds on existing training-free refinement methods.
The paper tackles the problem of text omission in diffusion-based text-to-image models by proposing TextGuider, a training-free method that aligns textual tokens with image regions using attention patterns and latent guidance, achieving state-of-the-art performance with significant gains in recall and strong OCR accuracy and CLIP scores.
Despite recent advances, diffusion-based text-to-image models still struggle with accurate text rendering. Several studies have proposed fine-tuning or training-free refinement methods for accurate text rendering. However, the critical issue of text omission, where the desired text is partially or entirely missing, remains largely overlooked. In this work, we propose TextGuider, a novel training-free method that encourages accurate and complete text appearance by aligning textual content tokens and text regions in the image. Specifically, we analyze attention patterns in MM-DiT models, particularly for text-related tokens intended to be rendered in the image. Leveraging this observation, we apply latent guidance during the early stage of denoising steps based on two loss functions that we introduce. Our method achieves state-of-the-art performance in test-time text rendering, with significant gains in recall and strong results in OCR accuracy and CLIP score.