DomainGallery: Few-shot Domain-driven Image Generation by Attribute-centric Finetuning
This work addresses a domain-specific problem for users needing tailored image generation, but it is incremental as it builds on existing models and techniques.
The authors tackled the problem of generating images in specific domains that are hard to describe or unseen by text-to-image models, proposing DomainGallery, a few-shot domain-driven method that finetunes Stable Diffusion with attribute-centric techniques, achieving superior performance in various scenarios.
The recent progress in text-to-image models pretrained on large-scale datasets has enabled us to generate various images as long as we provide a text prompt describing what we want. Nevertheless, the availability of these models is still limited when we expect to generate images that fall into a specific domain either hard to describe or just unseen to the models. In this work, we propose DomainGallery, a few-shot domain-driven image generation method which aims at finetuning pretrained Stable Diffusion on few-shot target datasets in an attribute-centric manner. Specifically, DomainGallery features prior attribute erasure, attribute disentanglement, regularization and enhancement. These techniques are tailored to few-shot domain-driven generation in order to solve key issues that previous works have failed to settle. Extensive experiments are given to validate the superior performance of DomainGallery on a variety of domain-driven generation scenarios. Codes are available at https://github.com/Ldhlwh/DomainGallery.