Single Image Iterative Subject-driven Generation and Editing
This addresses the problem of low-quality personalization in image generation for users with limited subject images, offering a plug-and-play solution without training, though it is incremental as it builds on existing optimization techniques.
The paper tackles the challenge of personalizing image generation and editing from a single image without training, presenting SISO, a training-free approach that iteratively optimizes similarity with the subject image, resulting in significant improvements in image quality, subject fidelity, and background preservation over existing methods.
Personalizing image generation and editing is particularly challenging when we only have a few images of the subject, or even a single image. A common approach to personalization is concept learning, which can integrate the subject into existing models relatively quickly, but produces images whose quality tends to deteriorate quickly when the number of subject images is small. Quality can be improved by pre-training an encoder, but training restricts generation to the training distribution, and is time consuming. It is still an open hard challenge to personalize image generation and editing from a single image without training. Here, we present SISO, a novel, training-free approach based on optimizing a similarity score with an input subject image. More specifically, SISO iteratively generates images and optimizes the model based on loss of similarity with the given subject image until a satisfactory level of similarity is achieved, allowing plug-and-play optimization to any image generator. We evaluated SISO in two tasks, image editing and image generation, using a diverse data set of personal subjects, and demonstrate significant improvements over existing methods in image quality, subject fidelity, and background preservation.