Perceptual Similarity guidance and text guidance optimization for Editing Real Images using Guided Diffusion Models
This addresses image editing fidelity for users of diffusion models, but it is incremental as it builds on existing guidance techniques.
The paper tackles the problem of maintaining fidelity to the original image when editing with diffusion models, resulting in a dual-guidance method that preserves unaltered areas while realistically rendering edited elements.
When using a diffusion model for image editing, there are times when the modified image can differ greatly from the source. To address this, we apply a dual-guidance approach to maintain high fidelity to the original in areas that are not altered. First, we employ text-guided optimization, using text embeddings to direct latent space and classifier-free guidance. Second, we use perceptual similarity guidance, optimizing latent vectors with posterior sampling via Tweedie formula during the reverse process. This method ensures the realistic rendering of both the edited elements and the preservation of the unedited parts of the original image.