PEO: Training-Free Aesthetic Quality Enhancement in Pre-Trained Text-to-Image Diffusion Models with Prompt Embedding Optimization
This addresses the problem of generating higher-quality images from simple prompts for users of text-to-image models, but it is incremental as it builds on existing diffusion models without introducing a new paradigm.
The paper tackles the problem of improving aesthetic quality in pre-trained text-to-image diffusion models given simple prompts, and the result is a training-free method called Prompt Embedding Optimization (PEO) that enhances visual quality by optimizing text embeddings, achieving performance that exceeds or equals state-of-the-art methods.
This paper introduces a novel approach to aesthetic quality improvement in pre-trained text-to-image diffusion models when given a simple prompt. Our method, dubbed Prompt Embedding Optimization (PEO), leverages a pre-trained text-to-image diffusion model as a backbone and optimizes the text embedding of a given simple and uncurated prompt to enhance the visual quality of the generated image. We achieve this by a tripartite objective function that improves the aesthetic fidelity of the generated image, ensures adherence to the optimized text embedding, and minimal divergence from the initial prompt. The latter is accomplished through a prompt preservation term. Additionally, PEO is training-free and backbone-independent. Quantitative and qualitative evaluations confirm the effectiveness of the proposed method, exceeding or equating the performance of state-of-the-art text-to-image and prompt adaptation methods.