CrimEdit: Controllable Editing for Counterfactual Object Removal, Insertion, and Movement
This addresses the challenge of composite image editing for users needing precise control over object effects, though it is incremental by building on existing diffusion-based methods.
The paper tackles the problem of handling object effects like shadows and reflections in image editing tasks such as removal, insertion, and movement, proposing CrimEdit which uses a unified model with classifier-free guidance to improve performance and efficiency. Results show superior removal, controllable effect insertion, and efficient movement without extra training or separate stages.
Recent works on object removal and insertion have enhanced their performance by handling object effects such as shadows and reflections, using diffusion models trained on counterfactual datasets. However, the performance impact of applying classifier-free guidance to handle object effects across removal and insertion tasks within a unified model remains largely unexplored. To address this gap and improve efficiency in composite editing, we propose CrimEdit, which jointly trains the task embeddings for removal and insertion within a single model and leverages them in a classifier-free guidance scheme -- enhancing the removal of both objects and their effects, and enabling controllable synthesis of object effects during insertion. CrimEdit also extends these two task prompts to be applied to spatially distinct regions, enabling object movement (repositioning) within a single denoising step. By employing both guidance techniques, extensive experiments show that CrimEdit achieves superior object removal, controllable effect insertion, and efficient object movement without requiring additional training or separate removal and insertion stages.