Customize Your Own Paired Data via Few-shot Way
This work addresses the problem of data scarcity and domain restrictions in image editing for users needing customizable effects, representing an incremental improvement over existing methods.
The paper tackles the problem of image editing by enabling users to customize effects with only a few image pairs, addressing limitations of supervised methods that require large paired datasets and unsupervised methods that rely on pre-trained priors. It introduces a novel few-shot learning mechanism based on directional transformations, achieving capabilities demonstrated in various experimental cases.
Existing solutions to image editing tasks suffer from several issues. Though achieving remarkably satisfying generated results, some supervised methods require huge amounts of paired training data, which greatly limits their usages. The other unsupervised methods take full advantage of large-scale pre-trained priors, thus being strictly restricted to the domains where the priors are trained on and behaving badly in out-of-distribution cases. The task we focus on is how to enable the users to customize their desired effects through only few image pairs. In our proposed framework, a novel few-shot learning mechanism based on the directional transformations among samples is introduced and expands the learnable space exponentially. Adopting a diffusion model pipeline, we redesign the condition calculating modules in our model and apply several technical improvements. Experimental results demonstrate the capabilities of our method in various cases.