MultiPrompter: Cooperative Prompt Optimization with Multi-Agent Reinforcement Learning
This work addresses a bottleneck in automated prompt optimization for users of black-box foundation models, offering an incremental improvement over existing RL methods.
The paper tackles the challenge of suboptimal policy convergence in RL-based prompt optimization by introducing MultiPrompter, a cooperative multi-agent framework that reduces problem size, resulting in higher-quality image generation compared to baselines.
Recently, there has been an increasing interest in automated prompt optimization based on reinforcement learning (RL). This approach offers important advantages, such as generating interpretable prompts and being compatible with black-box foundation models. However, the substantial prompt space size poses challenges for RL-based methods, often leading to suboptimal policy convergence. This paper introduces MultiPrompter, a new framework that views prompt optimization as a cooperative game between prompters which take turns composing a prompt together. Our cooperative prompt optimization effectively reduces the problem size and helps prompters learn optimal prompts. We test our method on the text-to-image task and show its ability to generate higher-quality images than baselines.