A Data Perspective on Enhanced Identity Preservation for Diffusion Personalization
This addresses the challenge of generating accurate personal images like pets or furniture for users of diffusion models, though it is an incremental improvement over existing methods.
The paper tackles the problem of preserving subject identity in personalized text-to-image generation by introducing a data-centric regularization strategy, achieving new state-of-the-art results on established benchmarks for identity preservation and text alignment.
Large text-to-image models have revolutionized the ability to generate imagery using natural language. However, particularly unique or personal visual concepts, such as pets and furniture, will not be captured by the original model. This has led to interest in how to personalize a text-to-image model. Despite significant progress, this task remains a formidable challenge, particularly in preserving the subject's identity. Most researchers attempt to address this issue by modifying model architectures. These methods are capable of keeping the subject structure and color but fail to preserve identity details. Towards this issue, our approach takes a data-centric perspective. We introduce a novel regularization dataset generation strategy on both the text and image level. This strategy enables the model to preserve fine details of the desired subjects, such as text and logos. Our method is architecture-agnostic and can be flexibly applied on various text-to-image models. We show on established benchmarks that our data-centric approach forms the new state of the art in terms of identity preservation and text alignment.