Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting
This addresses the problem for users who struggle to articulate ideas effectively in text-to-image systems, though it is incremental as it builds on existing prompt enhancement methods.
The paper tackles the challenge of creating personalized visual representations in text-to-image generation by rewriting user prompts based on historical interactions, using a dataset of over 300k prompts from 3115 users, and shows superiority over baselines in offline and online tests.
Despite significant progress in the field, it is still challenging to create personalized visual representations that align closely with the desires and preferences of individual users. This process requires users to articulate their ideas in words that are both comprehensible to the models and accurately capture their vision, posing difficulties for many users. In this paper, we tackle this challenge by leveraging historical user interactions with the system to enhance user prompts. We propose a novel approach that involves rewriting user prompts based on a newly collected large-scale text-to-image dataset with over 300k prompts from 3115 users. Our rewriting model enhances the expressiveness and alignment of user prompts with their intended visual outputs. Experimental results demonstrate the superiority of our methods over baseline approaches, as evidenced in our new offline evaluation method and online tests. Our code and dataset are available at https://github.com/zzjchen/Tailored-Visions.