XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation
This addresses the problem of personalized and complex scene generation for users of text-to-image models, representing a novel method for a known bottleneck rather than a foundational advancement.
The paper tackles the problem of achieving fine-grained control over subject identity and semantic attributes in multi-subject text-to-image generation with Diffusion Transformers, which often causes artifacts or attribute entanglement. The result is XVerse, a model that transforms reference images into offsets for token-specific text-stream modulation, enabling high-fidelity, editable multi-subject image synthesis with robust control.
Achieving fine-grained control over subject identity and semantic attributes (pose, style, lighting) in text-to-image generation, particularly for multiple subjects, often undermines the editability and coherence of Diffusion Transformers (DiTs). Many approaches introduce artifacts or suffer from attribute entanglement. To overcome these challenges, we propose a novel multi-subject controlled generation model XVerse. By transforming reference images into offsets for token-specific text-stream modulation, XVerse allows for precise and independent control for specific subject without disrupting image latents or features. Consequently, XVerse offers high-fidelity, editable multi-subject image synthesis with robust control over individual subject characteristics and semantic attributes. This advancement significantly improves personalized and complex scene generation capabilities.