TIPO: Text to Image with Text Presampling for Prompt Optimization
This addresses the need for efficient and scalable prompt optimization in text-to-image tasks, offering a computationally lighter alternative to methods based on large language models or reinforcement learning, though it is incremental in improving existing prompt engineering approaches.
The paper tackles the problem of automatic prompt refinement for text-to-image generation by introducing TIPO, which expands simple user prompts into richer versions, resulting in substantial improvements in aesthetic quality, reduction of visual artifacts, and enhanced alignment with target distributions, as demonstrated by experimental evaluations and human preference reports.
TIPO (Text-to-Image Prompt Optimization) introduces an efficient approach for automatic prompt refinement in text-to-image (T2I) generation. Starting from simple user prompts, TIPO leverages a lightweight pre-trained model to expand these prompts into richer, detailed versions. Conceptually, TIPO samples refined prompts from a targeted sub-distribution within the broader semantic space, preserving the original intent while significantly improving visual quality, coherence, and detail. Unlike resource-intensive methods based on large language models (LLMs) or reinforcement learning (RL), TIPO provides computational efficiency and scalability, opening new possibilities for effective, automated prompt engineering in T2I tasks. We provide visual results, human preference report to investigate TIPO's effectiveness. Experimental evaluations on benchmark datasets demonstrate substantial improvements in aesthetic quality, significant reduction of visual artifacts, and enhanced alignment with target distributions along with significant human preference proficiency. These results highlight the importance of targeted prompt engineering in text-to-image tasks and indicate broader opportunities for automated prompt refinement.