Text-Driven Image Editing via Learnable Regions
This addresses the challenge of flexible and user-friendly image editing for creative applications, though it is incremental as it builds on existing pre-trained models.
The paper tackles the problem of text-driven image editing without requiring user-provided masks or sketches by introducing a bounding box generator to identify editing regions aligned with textual prompts, achieving competitive performance in user studies with high fidelity and realism.
Language has emerged as a natural interface for image editing. In this paper, we introduce a method for region-based image editing driven by textual prompts, without the need for user-provided masks or sketches. Specifically, our approach leverages an existing pre-trained text-to-image model and introduces a bounding box generator to identify the editing regions that are aligned with the textual prompts. We show that this simple approach enables flexible editing that is compatible with current image generation models, and is able to handle complex prompts featuring multiple objects, complex sentences, or lengthy paragraphs. We conduct an extensive user study to compare our method against state-of-the-art methods. The experiments demonstrate the competitive performance of our method in manipulating images with high fidelity and realism that correspond to the provided language descriptions. Our project webpage can be found at: https://yuanze-lin.me/LearnableRegions_page.