LLM-Enabled Style and Content Regularization for Personalized Text-to-Image Generation
This work addresses the challenge of maintaining textual controllability while improving stylization for users of personalized text-to-image models, representing an incremental advancement.
The paper tackled the problem of insufficient stylization and inaccurate content in personalized text-to-image generation by proposing style refinement and content preservation strategies, achieving superior performance in generating consistent and personalized outputs.
The personalized text-to-image generation has rapidly advanced with the emergence of Stable Diffusion. Existing methods, which typically fine-tune models using embedded identifiers, often struggle with insufficient stylization and inaccurate image content due to reduced textual controllability. In this paper, we propose style refinement and content preservation strategies. The style refinement strategy leverages the semantic information of visual reasoning prompts and reference images to optimize style embeddings, allowing a more precise and consistent representation of style information. The content preservation strategy addresses the content bias problem by preserving the model's generalization capabilities, ensuring enhanced textual controllability without compromising stylization. Experimental results verify that our approach achieves superior performance in generating consistent and personalized text-to-image outputs.