User-friendly Image Editing with Minimal Text Input: Leveraging Captioning and Injection Techniques
This addresses a usability issue for users of image editing tools by reducing the need for detailed text descriptions, though it is incremental as it builds on existing prompt generation techniques.
The paper tackles the problem of labor-intensive prompt engineering in text-driven image editing by proposing methods that combine prompt generation frameworks to make the process more user-friendly, achieving results comparable to ground-truth prompts.
Recent text-driven image editing in diffusion models has shown remarkable success. However, the existing methods assume that the user's description sufficiently grounds the contexts in the source image, such as objects, background, style, and their relations. This assumption is unsuitable for real-world applications because users have to manually engineer text prompts to find optimal descriptions for different images. From the users' standpoint, prompt engineering is a labor-intensive process, and users prefer to provide a target word for editing instead of a full sentence. To address this problem, we first demonstrate the importance of a detailed text description of the source image, by dividing prompts into three categories based on the level of semantic details. Then, we propose simple yet effective methods by combining prompt generation frameworks, thereby making the prompt engineering process more user-friendly. Extensive qualitative and quantitative experiments demonstrate the importance of prompts in text-driven image editing and our method is comparable to ground-truth prompts.