Understanding the Impact of Negative Prompts: When and How Do They Take Effect?
This addresses a gap in understanding for users and developers of generative AI models, though it is incremental as it builds on existing prompt-based methods.
The paper tackled the problem of understanding the intrinsic mechanisms of negative prompts in conditional generation models like Stable Diffusion, identifying two primary behaviors: delayed effect and deletion through neutralization, and demonstrating applications such as object inpainting with minimal background alterations.
The concept of negative prompts, emerging from conditional generation models like Stable Diffusion, allows users to specify what to exclude from the generated images.%, demonstrating significant practical efficacy. Despite the widespread use of negative prompts, their intrinsic mechanisms remain largely unexplored. This paper presents the first comprehensive study to uncover how and when negative prompts take effect. Our extensive empirical analysis identifies two primary behaviors of negative prompts. Delayed Effect: The impact of negative prompts is observed after positive prompts render corresponding content. Deletion Through Neutralization: Negative prompts delete concepts from the generated image through a mutual cancellation effect in latent space with positive prompts. These insights reveal significant potential real-world applications; for example, we demonstrate that negative prompts can facilitate object inpainting with minimal alterations to the background via a simple adaptive algorithm. We believe our findings will offer valuable insights for the community in capitalizing on the potential of negative prompts.