Prompt Evolution for Generative AI: A Classifier-Guided Approach
This addresses the limitation of generative AI in accurately reflecting user preferences, offering a novel method for improving output fidelity in applications like image synthesis.
The paper tackles the problem of generative AI models failing to connect generated outputs with desired target concepts in user prompts by proposing prompt evolution, which uses evolutionary selection pressure and variation during generation to produce multiple outputs that better satisfy preferences, resulting in more faithful and diversified images.
Synthesis of digital artifacts conditioned on user prompts has become an important paradigm facilitating an explosion of use cases with generative AI. However, such models often fail to connect the generated outputs and desired target concepts/preferences implied by the prompts. Current research addressing this limitation has largely focused on enhancing the prompts before output generation or improving the model's performance up front. In contrast, this paper conceptualizes prompt evolution, imparting evolutionary selection pressure and variation during the generative process to produce multiple outputs that satisfy the target concepts/preferences better. We propose a multi-objective instantiation of this broader idea that uses a multi-label image classifier-guided approach. The predicted labels from the classifiers serve as multiple objectives to optimize, with the aim of producing diversified images that meet user preferences. A novelty of our evolutionary algorithm is that the pre-trained generative model gives us implicit mutation operations, leveraging the model's stochastic generative capability to automate the creation of Pareto-optimized images more faithful to user preferences.