Taming Text-to-Image Synthesis for Novices: User-centric Prompt Generation via Multi-turn Guidance
This work addresses usability issues for novice users in text-to-image synthesis, though it is incremental as it builds on existing prompt generation methods by adding interactive dialogue.
The paper tackles the challenge of novice users struggling with text-to-image synthesis prompt writing by proposing DialPrompt, a dialogue-based model that guides users through multi-turn interactions to generate prompts, resulting in significantly improved user-centricity scores while maintaining competitive image quality.
The emergence of text-to-image synthesis (TIS) models has significantly influenced digital image creation by producing high-quality visuals from written descriptions. Yet these models are sensitive on textual prompts, posing a challenge for novice users who may not be familiar with TIS prompt writing. Existing solutions relieve this via automatic prompt expansion or generation from a user query. However, this single-turn manner suffers from limited user-centricity in terms of result interpretability and user interactivity. Thus, we propose DialPrompt, a dialogue-based TIS prompt generation model that emphasizes user experience for novice users. DialPrompt is designed to follow a multi-turn workflow, where in each round of dialogue the model guides user to express their preferences on possible optimization dimensions before generating the final TIS prompt. To achieve this, we mined 15 essential dimensions for high-quality prompts from advanced users and curated a multi-turn dataset. Through training on this dataset, DialPrompt improves user-centricity by allowing users to perceive and control the creation process of TIS prompts. Experiments indicate that DialPrompt improves significantly in user-centricity score compared with existing approaches while maintaining a competitive quality of synthesized images. In our user evaluation, DialPrompt is highly rated by 19 human reviewers (especially novices).