Investigating Prompt Engineering in Diffusion Models
This addresses a practical problem for artists using diffusion models, but it is incremental as it focuses on measuring and guiding prompt selection rather than introducing new methods.
The paper tackles the challenge of selecting effective prompts for text-to-image diffusion models by presenting techniques to measure the impact of specific words and phrases, and provides guidance for artists to achieve desired artistic outputs.
With the spread of the use of Text2Img diffusion models such as DALL-E 2, Imagen, Mid Journey and Stable Diffusion, one challenge that artists face is selecting the right prompts to achieve the desired artistic output. We present techniques for measuring the effect that specific words and phrases in prompts have, and (in the Appendix) present guidance on the selection of prompts to produce desired effects.