TextInVision: Text and Prompt Complexity Driven Visual Text Generation Benchmark
This addresses the lack of comprehensive benchmarks for text embedding in AI-generated images, which is crucial for applications like educational materials and advertisements, though it is incremental as it focuses on evaluation rather than new model development.
The authors tackled the problem of evaluating diffusion-based text-to-image models' ability to embed text accurately in images by introducing TextInVision, a large-scale benchmark that identifies common errors like spelling inaccuracies and contextual mismatches.
Generating images with embedded text is crucial for the automatic production of visual and multimodal documents, such as educational materials and advertisements. However, existing diffusion-based text-to-image models often struggle to accurately embed text within images, facing challenges in spelling accuracy, contextual relevance, and visual coherence. Evaluating the ability of such models to embed text within a generated image is complicated due to the lack of comprehensive benchmarks. In this work, we introduce TextInVision, a large-scale, text and prompt complexity driven benchmark designed to evaluate the ability of diffusion models to effectively integrate visual text into images. We crafted a diverse set of prompts and texts that consider various attributes and text characteristics. Additionally, we prepared an image dataset to test Variational Autoencoder (VAE) models across different character representations, highlighting that VAE architectures can also pose challenges in text generation within diffusion frameworks. Through extensive analysis of multiple models, we identify common errors and highlight issues such as spelling inaccuracies and contextual mismatches. By pinpointing the failure points across different prompts and texts, our research lays the foundation for future advancements in AI-generated multimodal content.