DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting
This work addresses a specific bottleneck in customized image inpainting for users needing precise object editing, representing an incremental improvement over existing methods.
The paper tackled the problem of identity overfitting in subject-driven image inpainting, where original object attributes interfere with text-guided edits, and proposed DreamMix, a diffusion-based framework that achieved a superior balance between identity preservation and attribute editability in experiments across multiple inpainting backbones.
Subject-driven image inpainting has recently gained prominence in image editing with the rapid advancement of diffusion models. Beyond image guidance, recent studies have explored incorporating text guidance to achieve identity-preserved yet locally editable object inpainting. However, these methods still suffer from identity overfitting, where original attributes remain entangled with target textual instructions. To overcome this limitation, we propose DreamMix, a diffusion-based framework adept at inserting target objects into user-specified regions while concurrently enabling arbitrary text-driven attribute modifications. DreamMix introduces three key components: (i) an Attribute Decoupling Mechanism (ADM) that synthesizes diverse attribute-augmented image-text pairs to mitigate overfitting; (ii) a Textual Attribute Substitution (TAS) module that isolates target attributes via orthogonal decomposition, and (iii) a Disentangled Inpainting Framework (DIF) that seperates local generation from global harmonization. Extensive experiments across multiple inpainting backbones demonstrate that DreamMix achieves a superior balance between identity preservation and attribute editability across diverse applications, including object insertion, attribute editing, and small object inpainting.