h-Edit: Effective and Flexible Diffusion-Based Editing via Doob's h-Transform
This addresses the need for flexible and effective image editing tools in computer vision, offering a novel theoretical framework and method for complex editing tasks.
The paper tackles the problem of diffusion-based image editing by introducing h-Edit, a training-free method that uses Doob's h-transform and Langevin Monte Carlo to decompose updates into reconstruction and editing terms, enabling simultaneous text-guided and reward-model-based editing and outperforming state-of-the-art baselines in effectiveness and faithfulness.
We introduce a theoretical framework for diffusion-based image editing by formulating it as a reverse-time bridge modeling problem. This approach modifies the backward process of a pretrained diffusion model to construct a bridge that converges to an implicit distribution associated with the editing target at time 0. Building on this framework, we propose h-Edit, a novel editing method that utilizes Doob's h-transform and Langevin Monte Carlo to decompose the update of an intermediate edited sample into two components: a "reconstruction" term and an "editing" term. This decomposition provides flexibility, allowing the reconstruction term to be computed via existing inversion techniques and enabling the combination of multiple editing terms to handle complex editing tasks. To our knowledge, h-Edit is the first training-free method capable of performing simultaneous text-guided and reward-model-based editing. Extensive experiments, both quantitative and qualitative, show that h-Edit outperforms state-of-the-art baselines in terms of editing effectiveness and faithfulness. Our source code is available at https://github.com/nktoan/h-edit.